[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:17 dist-test-slave-z2t3 krb5kdc[5312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:17 dist-test-slave-z2t3 krb5kdc[5312](info): set up 2 sockets
Sep 28 17:56:17 dist-test-slave-z2t3 krb5kdc[5312](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 17:56:18.153457 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:18.279520 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34227
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:34227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:18.362643 5325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:18.362926 5325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:18.362969 5325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:18.364283 5325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:18.364362 5325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:18.364387 5325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:18.364406 5325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:18.364439 5325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:18.366003 5325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:34227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:18.366294 5325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:18.366609 5325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:18.372112 5325 server_base.cc:1034] running on GCE node
W20260928 17:56:18.373070 5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:18.373497 5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:18.373526 5331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.373890 5325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:18.374219 5325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:18.375425 5325 hybrid_clock.cc:648] HybridClock initialized: now 1790618178375402 us; error 38 us; skew 500 ppm
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:18.378206 5325 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:18.378834 5325 webserver.cc:460] Webserver started at http://127.5.46.126:38809/ using document root <none> and password file <none>
I20260928 17:56:18.379070 5325 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:18.379139 5325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:18.379302 5325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:18.380533 5325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "fc7ecbe530e0482081145c3644a88663"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "b237531a67bc423aa92e6d925a21c0d8"
server_key_iv: "56ed4e496e1a0ca1030bf2cf4dee9e2d"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.380939 5325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "fc7ecbe530e0482081145c3644a88663"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "b237531a67bc423aa92e6d925a21c0d8"
server_key_iv: "56ed4e496e1a0ca1030bf2cf4dee9e2d"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.382440 5325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 17:56:18.383364 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.383605 5325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:18.383719 5325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "fc7ecbe530e0482081145c3644a88663"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "b237531a67bc423aa92e6d925a21c0d8"
server_key_iv: "56ed4e496e1a0ca1030bf2cf4dee9e2d"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.383800 5325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:18.394094 5325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:18.396347 5325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:18.396538 5325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:18.400787 5325 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:34227
I20260928 17:56:18.400820 5392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:34227 every 8 connection(s)
I20260928 17:56:18.401202 5325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:18.402180 5393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:18.404795 5393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663: Bootstrap starting.
I20260928 17:56:18.404786 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5325
I20260928 17:56:18.404901 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:18.405555 5393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:18.405689 5305 external_mini_cluster.cc:1428] Setting key 981d79304d966810830447b8700beaf2
I20260928 17:56:18.405934 5393 log.cc:824] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:18.406884 5393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663: No bootstrap required, opened a new log
I20260928 17:56:18.408741 5393 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } }
I20260928 17:56:18.408896 5393 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:18.408919 5393 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7ecbe530e0482081145c3644a88663, State: Initialized, Role: FOLLOWER
I20260928 17:56:18.409060 5393 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } }
I20260928 17:56:18.409168 5393 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:18.409221 5393 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:18.409296 5393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:18.410301 5393 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } }
I20260928 17:56:18.410434 5393 leader_election.cc:302] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7ecbe530e0482081145c3644a88663; no voters:
I20260928 17:56:18.410646 5393 leader_election.cc:288] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:18.410677 5398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:18.410794 5398 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [term 1 LEADER]: Becoming Leader. State: Replica: fc7ecbe530e0482081145c3644a88663, State: Running, Role: LEADER
I20260928 17:56:18.410928 5393 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:18.410935 5398 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } }
I20260928 17:56:18.411401 5400 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7ecbe530e0482081145c3644a88663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } } }
I20260928 17:56:18.411414 5399 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7ecbe530e0482081145c3644a88663. Latest consensus state: current_term: 1 leader_uuid: "fc7ecbe530e0482081145c3644a88663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7ecbe530e0482081145c3644a88663" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34227 } } }
I20260928 17:56:18.411576 5400 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:18.411675 5399 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:18.411991 5407 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:18.412531 5407 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:18.414062 5407 catalog_manager.cc:1347] Generated new cluster ID: 819b436daef44ddfa093ceaa9f7a419c
I20260928 17:56:18.414112 5407 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:18.424669 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.406852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.5.46.126:34227)
0928 17:56:18.407041 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:18.407052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:18.407388 (+ 336us) server_negotiation.cc:407] Connection header received
0928 17:56:18.409244 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:18.409259 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:18.409313 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:18.409450 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:18.411446 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.412889 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.414358 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.414934 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.415037 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.419594 (+ 4557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:18.419600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:18.419600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:18.419614 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:18.422320 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.422864 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.422868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.422868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.422986 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.423279 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.423280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.423280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.423577 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:18.423725 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:18.424066 (+ 341us) server_negotiation.cc:299] Negotiation successful
0928 17:56:18.424205 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260928 17:56:18.442544 5407 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:18.443145 5407 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:18.448843 5407 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7ecbe530e0482081145c3644a88663: Generated new TSK 0
I20260928 17:56:18.449079 5407 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:18.484361 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:18.574244 5419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:18.574503 5419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:18.574534 5419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:18.575982 5419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:18.576037 5419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:18.576071 5419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:18.577793 5419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:18.578038 5419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:18.578258 5419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:18.584105 5427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.584122 5419 server_base.cc:1034] running on GCE node
W20260928 17:56:18.584210 5425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:18.584105 5424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.584640 5419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:18.584885 5419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:18.586091 5419 hybrid_clock.cc:648] HybridClock initialized: now 1790618178586077 us; error 43 us; skew 500 ppm
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:18.593245 5419 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:18.593771 5419 webserver.cc:460] Webserver started at http://127.5.46.65:38485/ using document root <none> and password file <none>
I20260928 17:56:18.593997 5419 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:18.594043 5419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:18.594141 5419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:18.595204 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "7f6614f50d4b4ac2bf2c9d5e5a0031e3"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "842e2d7ac148e8c5df9bf84918a5c23a"
server_key_iv: "75f088f4c50f226e0f84d2681fc24c28"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.595535 5419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "7f6614f50d4b4ac2bf2c9d5e5a0031e3"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "842e2d7ac148e8c5df9bf84918a5c23a"
server_key_iv: "75f088f4c50f226e0f84d2681fc24c28"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.597008 5419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:18.597756 5434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.597956 5419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.598047 5419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "7f6614f50d4b4ac2bf2c9d5e5a0031e3"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "842e2d7ac148e8c5df9bf84918a5c23a"
server_key_iv: "75f088f4c50f226e0f84d2681fc24c28"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.598104 5419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:18.613157 5419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:18.615631 5419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:18.615754 5419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:18.616032 5419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:18.616362 5419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:18.616397 5419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.616449 5419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:18.616474 5419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.622748 5419 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:43883
I20260928 17:56:18.622856 5547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:43883 every 8 connection(s)
I20260928 17:56:18.623178 5419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:18.630774 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5419
I20260928 17:56:18.630853 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:18.631013 5305 external_mini_cluster.cc:1428] Setting key ae040750eb62c2eff5b1d263328fe810
I20260928 17:56:18.634207 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.624120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:59085 (local address 127.5.46.126:34227)
0928 17:56:18.624219 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:18.624221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:18.624476 (+ 255us) server_negotiation.cc:407] Connection header received
0928 17:56:18.625050 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:18.625051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:18.625068 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:18.625130 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:18.626244 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.626813 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.627954 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.628176 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.628259 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.630663 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:18.630664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:18.630665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:18.630673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:18.632608 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.633114 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.633115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.633116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.633193 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.633517 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.633518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.633518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.633629 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:18.633682 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:18.634068 (+ 386us) server_negotiation.cc:299] Negotiation successful
0928 17:56:18.634139 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 17:56:18.634585 5550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.624078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34227 (local address 127.5.46.65:59085)
0928 17:56:18.624331 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:18.624372 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:18.624930 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:18.625255 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:18.625259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:18.625905 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:18.626124 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.626129 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.626927 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.626928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.627831 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.627832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.628288 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.628289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.628425 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.628953 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:18.628962 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:18.630540 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:18.632710 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.632718 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.632720 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.632995 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.633270 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.633271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.633271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.633444 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.633698 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:18.633704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:18.633914 (+ 210us) client_negotiation.cc:769] Sending connection context
0928 17:56:18.634050 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 17:56:18.634264 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260928 17:56:18.635176 5548 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34227
I20260928 17:56:18.635295 5548 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:18.635512 5548 heartbeater.cc:511] Master 127.5.46.126:34227 requested a full tablet report, sending...
I20260928 17:56:18.636193 5357 ts_manager.cc:194] Registered new tserver with Master: 7f6614f50d4b4ac2bf2c9d5e5a0031e3 (127.5.46.65:43883)
I20260928 17:56:18.636840 5357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:59085
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:18.660073 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:18.747655 5553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:18.747975 5553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:18.748009 5553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:18.749284 5553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:18.749338 5553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:18.749368 5553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:18.750921 5553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:18.751220 5553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:18.751513 5553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:18.756670 5559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.756745 5553 server_base.cc:1034] running on GCE node
W20260928 17:56:18.756834 5558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:18.756686 5561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.757079 5553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:18.757333 5553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:18.758481 5553 hybrid_clock.cc:648] HybridClock initialized: now 1790618178758460 us; error 40 us; skew 500 ppm
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:18.761232 5553 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:18.761808 5553 webserver.cc:460] Webserver started at http://127.5.46.66:38085/ using document root <none> and password file <none>
I20260928 17:56:18.762072 5553 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:18.762204 5553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:18.762360 5553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:18.763446 5553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "70f80380cf20485486ccac6c1b7ea643"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7297b6f5003d97d669fbf50b6e1c4397"
server_key_iv: "295762116afa99ce5516abd59cf21f47"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.763823 5553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "70f80380cf20485486ccac6c1b7ea643"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7297b6f5003d97d669fbf50b6e1c4397"
server_key_iv: "295762116afa99ce5516abd59cf21f47"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.765317 5553 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:18.766165 5568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.766351 5553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:18.766458 5553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "70f80380cf20485486ccac6c1b7ea643"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7297b6f5003d97d669fbf50b6e1c4397"
server_key_iv: "295762116afa99ce5516abd59cf21f47"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.766534 5553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:18.783595 5553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:18.785851 5553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:18.786043 5553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:18.786307 5553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:18.786643 5553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:18.786698 5553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.786744 5553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:18.786777 5553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.793124 5553 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:32905
I20260928 17:56:18.793170 5681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:32905 every 8 connection(s)
I20260928 17:56:18.793540 5553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:18.795543 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5553
I20260928 17:56:18.795662 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:18.795873 5305 external_mini_cluster.cc:1428] Setting key 58bd9cdf2a17bdfc43d1df21443669bd
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 17:56:18.804826 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.794364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48229 (local address 127.5.46.126:34227)
0928 17:56:18.794463 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:18.794464 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:18.794689 (+ 225us) server_negotiation.cc:407] Connection header received
0928 17:56:18.795236 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:18.795238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:18.795249 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:18.795352 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:18.796899 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.797444 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.798751 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.798966 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.799058 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.801695 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:18.801697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:18.801698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:18.801708 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:18.803248 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.803725 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.803727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.803728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.803798 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.804107 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.804107 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.804108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.804244 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:18.804319 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:18.804632 (+ 313us) server_negotiation.cc:299] Negotiation successful
0928 17:56:18.804727 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260928 17:56:18.805047 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.794347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34227 (local address 127.5.46.66:48229)
0928 17:56:18.794552 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:18.794596 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:18.795124 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:18.795494 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:18.795499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:18.796465 (+ 966us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:18.796772 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.796779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.797549 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.797550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.798630 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.798632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.799068 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.799068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.799231 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.799799 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:18.799810 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:18.801535 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:18.803368 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.803378 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.803381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.803641 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.803894 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.803895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.803896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.804031 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.804321 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:18.804327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:18.804483 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 17:56:18.804598 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 17:56:18.804790 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":130,"threads_started":1}
I20260928 17:56:18.805687 5682 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34227
I20260928 17:56:18.805819 5682 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:18.806030 5682 heartbeater.cc:511] Master 127.5.46.126:34227 requested a full tablet report, sending...
I20260928 17:56:18.806600 5357 ts_manager.cc:194] Registered new tserver with Master: 70f80380cf20485486ccac6c1b7ea643 (127.5.46.66:32905)
I20260928 17:56:18.807081 5357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:48229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:18.827739 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:18.913946 5687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:18.914156 5687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:18.914181 5687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:18.915385 5687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:18.915431 5687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:18.915464 5687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:18.917006 5687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:18.917261 5687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:18.917502 5687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:18.923475 5693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:18.923509 5692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.923537 5687 server_base.cc:1034] running on GCE node
W20260928 17:56:18.923471 5695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:18.923946 5687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:18.924275 5687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:18.925473 5687 hybrid_clock.cc:648] HybridClock initialized: now 1790618178925446 us; error 43 us; skew 500 ppm
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:18.928383 5687 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:18.928859 5687 webserver.cc:460] Webserver started at http://127.5.46.67:33125/ using document root <none> and password file <none>
I20260928 17:56:18.929101 5687 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:18.929170 5687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:18.929349 5687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:18.930477 5687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "75eaa1c330384bbabb038e84a8a4e5f6"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7e608a36727228a9b4b9524514289e1c"
server_key_iv: "8486fda1f7d9430b3c12e7e8856f3c12"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.930847 5687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "75eaa1c330384bbabb038e84a8a4e5f6"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7e608a36727228a9b4b9524514289e1c"
server_key_iv: "8486fda1f7d9430b3c12e7e8856f3c12"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.932287 5687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:18.933137 5702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.933377 5687 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:18.933490 5687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "75eaa1c330384bbabb038e84a8a4e5f6"
format_stamp: "Formatted at 2026-09-28 17:56:18 on dist-test-slave-z2t3"
server_key: "7e608a36727228a9b4b9524514289e1c"
server_key_iv: "8486fda1f7d9430b3c12e7e8856f3c12"
server_key_version: "encryptionkey@0"
I20260928 17:56:18.933568 5687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:18.952406 5687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:18.954983 5687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:18.955159 5687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:18.955415 5687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:18.955725 5687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:18.955778 5687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.955813 5687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:18.955888 5687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:18.962543 5687 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:41329
I20260928 17:56:18.962620 5815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:41329 every 8 connection(s)
I20260928 17:56:18.962971 5687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:18.963081 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5687
I20260928 17:56:18.963160 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:18.963333 5305 external_mini_cluster.cc:1428] Setting key 544aa01c585802839e93786f3e02b436
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:18.972815 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.963795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:57201 (local address 127.5.46.126:34227)
0928 17:56:18.963872 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:18.963875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:18.964139 (+ 264us) server_negotiation.cc:407] Connection header received
0928 17:56:18.964811 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:18.964812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:18.964824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:18.964880 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:18.965952 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.966466 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.967485 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.967696 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.967773 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.970015 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:18.970016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:18.970017 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:18.970023 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:18.971403 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.971823 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.971824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.971825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.971964 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.972219 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.972220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.972221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.972369 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:18.972433 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:18.972677 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 17:56:18.972746 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:18.973026 5818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.963807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34227 (local address 127.5.46.67:57201)
0928 17:56:18.964018 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:18.964066 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:18.964700 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:18.964955 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:18.964959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:18.965646 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:18.965854 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.965859 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.966563 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.966564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.967378 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:18.967380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.967786 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.967787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:18.967965 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.968473 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:18.968483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:18.969901 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:18.971484 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.971489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.971491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.971735 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.972036 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:18.972036 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:18.972037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:18.972138 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:18.972434 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:18.972438 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:18.972559 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 17:56:18.972646 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 17:56:18.972769 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260928 17:56:18.973573 5816 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34227
I20260928 17:56:18.973657 5816 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:18.973817 5816 heartbeater.cc:511] Master 127.5.46.126:34227 requested a full tablet report, sending...
I20260928 17:56:18.974161 5357 ts_manager.cc:194] Registered new tserver with Master: 75eaa1c330384bbabb038e84a8a4e5f6 (127.5.46.67:41329)
I20260928 17:56:18.974530 5357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:57201
I20260928 17:56:18.975579 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 17:56:18 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618178, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:18.998776 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:18.990818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54522 (local address 127.5.46.65:43883)
0928 17:56:18.990975 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:18.990980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:18.990993 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:56:18.991024 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:18.991026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:18.991043 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:18.991185 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:18.991764 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.992574 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.993332 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:18.993679 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:18.993780 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:18.996151 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:18.996165 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:18.996165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:18.996172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:18.997568 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.997931 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.997941 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.997942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.997993 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:18.998192 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:18.998193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:18.998194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:18.998334 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:18.998511 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:18.998566 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 17:56:18.998646 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
W20260928 17:56:18.999135 5462 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54522
Sep 28 17:56:19 dist-test-slave-z2t3 krb5kdc[5312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 17:56:19 dist-test-slave-z2t3 krb5kdc[5312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618179, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:19.022465 5823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:19.015050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54524 (local address 127.5.46.65:43883)
0928 17:56:19.015159 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:19.015161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:19.015322 (+ 161us) server_negotiation.cc:407] Connection header received
0928 17:56:19.015393 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:19.015394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:19.015407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:19.015461 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:19.016102 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:19.016609 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:19.017336 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:19.017537 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:19.017590 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:19.020036 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:19.020038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:19.020038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:19.020045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:19.021416 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:19.021758 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:19.021759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:19.021760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:19.021810 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:19.021987 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:19.021988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:19.021988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:19.022078 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:19.022163 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:19.022241 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 17:56:19.022336 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260928 17:56:19.023033 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5419
I20260928 17:56:19.028326 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5553
I20260928 17:56:19.033474 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5687
I20260928 17:56:19.037889 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5325
2026-09-28T17:56:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1954 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:19 dist-test-slave-z2t3 krb5kdc[5830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:19 dist-test-slave-z2t3 krb5kdc[5830](info): set up 2 sockets
Sep 28 17:56:19 dist-test-slave-z2t3 krb5kdc[5830](info): commencing operation
krb5kdc: starting...
W20260928 17:56:20.075659 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:20.208413 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35639
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:35639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:20.290797 5843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:20.291077 5843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:20.291118 5843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:20.292495 5843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:20.292572 5843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:20.292596 5843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:20.292618 5843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:20.292647 5843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:20.294194 5843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:35639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:20.294503 5843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:20.294749 5843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:20.300228 5849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.300210 5851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:20.300396 5843 server_base.cc:1034] running on GCE node
W20260928 17:56:20.300210 5848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:20.300724 5843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:20.301074 5843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:20.302261 5843 hybrid_clock.cc:648] HybridClock initialized: now 1790618180302241 us; error 43 us; skew 500 ppm
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:20.305071 5843 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:20.305688 5843 webserver.cc:460] Webserver started at http://127.5.46.126:44655/ using document root <none> and password file <none>
I20260928 17:56:20.305943 5843 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:20.306012 5843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:20.306213 5843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:20.307336 5843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "0a143b3697294169885ecc3005aebfc5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "cfec504cab4fd333b900b5e0f9fa7574"
server_key_iv: "89504d5efe76d361d9f4e957e272645e"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.307723 5843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "0a143b3697294169885ecc3005aebfc5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "cfec504cab4fd333b900b5e0f9fa7574"
server_key_iv: "89504d5efe76d361d9f4e957e272645e"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.309267 5843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:20.310174 5858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.310469 5843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:20.310590 5843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "0a143b3697294169885ecc3005aebfc5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "cfec504cab4fd333b900b5e0f9fa7574"
server_key_iv: "89504d5efe76d361d9f4e957e272645e"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.310673 5843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:20.324486 5843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:20.326974 5843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:20.327150 5843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:20.331368 5843 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:35639
I20260928 17:56:20.331424 5910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:35639 every 8 connection(s)
I20260928 17:56:20.331804 5843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:20.332768 5911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:20.333592 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5843
I20260928 17:56:20.333698 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:20.333876 5305 external_mini_cluster.cc:1428] Setting key e5c67a668165f919932a9fcad3d05f5e
I20260928 17:56:20.336109 5911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5: Bootstrap starting.
I20260928 17:56:20.336831 5911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:20.337142 5911 log.cc:824] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:20.337895 5911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5: No bootstrap required, opened a new log
I20260928 17:56:20.340169 5911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } }
I20260928 17:56:20.340324 5911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:20.340349 5911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a143b3697294169885ecc3005aebfc5, State: Initialized, Role: FOLLOWER
I20260928 17:56:20.340499 5911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } }
I20260928 17:56:20.340585 5911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:20.340616 5911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:20.340667 5911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:20.341457 5911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } }
I20260928 17:56:20.341576 5911 leader_election.cc:302] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a143b3697294169885ecc3005aebfc5; no voters:
I20260928 17:56:20.341795 5911 leader_election.cc:288] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:20.341864 5916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:20.342064 5916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [term 1 LEADER]: Becoming Leader. State: Replica: 0a143b3697294169885ecc3005aebfc5, State: Running, Role: LEADER
I20260928 17:56:20.342132 5911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:20.342191 5916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } }
I20260928 17:56:20.342629 5916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a143b3697294169885ecc3005aebfc5. Latest consensus state: current_term: 1 leader_uuid: "0a143b3697294169885ecc3005aebfc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } } }
I20260928 17:56:20.342636 5918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a143b3697294169885ecc3005aebfc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a143b3697294169885ecc3005aebfc5" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35639 } } }
I20260928 17:56:20.342769 5916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:20.342777 5918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:20.343011 5925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:20.343786 5925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:20.345799 5925 catalog_manager.cc:1347] Generated new cluster ID: 6917af3526fd4417a2b1540893b8fbba
I20260928 17:56:20.345854 5925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:20.346640 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.334473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.5.46.126:35639)
0928 17:56:20.334672 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.334683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.334716 (+ 33us) server_negotiation.cc:407] Connection header received
0928 17:56:20.335216 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.335226 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.335297 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.335454 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.336070 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.337461 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.338289 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.338918 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.339054 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.341734 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.341739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.341740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.341754 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.344520 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.344997 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.345000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.345001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.345104 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.345321 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.345323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.345323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.345644 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.345797 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.346253 (+ 456us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.346440 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260928 17:56:20.360632 5925 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:20.361199 5925 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:20.366508 5925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a143b3697294169885ecc3005aebfc5: Generated new TSK 0
I20260928 17:56:20.366772 5925 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:20.399576 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:20.491062 5937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:20.491344 5937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:20.491376 5937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:20.492965 5937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:20.493034 5937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:20.493085 5937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:20.495007 5937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:20.495271 5937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:20.495476 5937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:20.501123 5945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.501116 5942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.501151 5943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:20.501441 5937 server_base.cc:1034] running on GCE node
I20260928 17:56:20.501681 5937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:20.501994 5937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:20.503163 5937 hybrid_clock.cc:648] HybridClock initialized: now 1790618180503152 us; error 32 us; skew 500 ppm
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:20.505942 5937 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:20.506574 5937 webserver.cc:460] Webserver started at http://127.5.46.65:36081/ using document root <none> and password file <none>
I20260928 17:56:20.506829 5937 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:20.506904 5937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:20.507063 5937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:20.508281 5937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "6d5f02625de34c7bb627d4b9b229f654"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "6f9d7150c6d122dffd4b8be76e0046e0"
server_key_iv: "482962f615ecbed7dc62b08cee91b3ee"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.508682 5937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "6d5f02625de34c7bb627d4b9b229f654"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "6f9d7150c6d122dffd4b8be76e0046e0"
server_key_iv: "482962f615ecbed7dc62b08cee91b3ee"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.510191 5937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:20.511036 5952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.511236 5937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:20.511344 5937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "6d5f02625de34c7bb627d4b9b229f654"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "6f9d7150c6d122dffd4b8be76e0046e0"
server_key_iv: "482962f615ecbed7dc62b08cee91b3ee"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.511423 5937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:20.529611 5937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:20.532300 5937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:20.532483 5937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:20.532730 5937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:20.533058 5937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:20.533113 5937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.533154 5937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:20.533180 5937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.539111 5937 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:46395
I20260928 17:56:20.539155 6065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:46395 every 8 connection(s)
I20260928 17:56:20.539490 5937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:20.544934 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 5937
I20260928 17:56:20.545065 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:20.545320 5305 external_mini_cluster.cc:1428] Setting key 45b75b7aecfb08f5d761a1cd442a6cca
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:20.550220 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.540323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:51711 (local address 127.5.46.126:35639)
0928 17:56:20.540405 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.540407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.540597 (+ 190us) server_negotiation.cc:407] Connection header received
0928 17:56:20.541592 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.541593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.541606 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.541670 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.542367 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.543049 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.544144 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.544355 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.544414 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.547101 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.547102 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.547102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.547109 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.548649 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.549101 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.549103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.549103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.549164 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.549522 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.549523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.549523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.549648 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.549725 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.550025 (+ 300us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.550115 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 17:56:20.550437 6068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.540281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35639 (local address 127.5.46.65:51711)
0928 17:56:20.540469 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.540507 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.541424 (+ 917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.541778 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.541781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.542061 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:20.542257 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.542262 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.543130 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.543132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.544058 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.544060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.544430 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.544431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.544542 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.545160 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:20.545171 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:20.547009 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:20.548764 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.548771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.548774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.549015 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.549314 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.549315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.549315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.549450 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.549725 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:20.549732 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:20.549883 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.549992 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.550163 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260928 17:56:20.550979 6066 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35639
I20260928 17:56:20.551120 6066 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:20.551347 6066 heartbeater.cc:511] Master 127.5.46.126:35639 requested a full tablet report, sending...
I20260928 17:56:20.551972 5875 ts_manager.cc:194] Registered new tserver with Master: 6d5f02625de34c7bb627d4b9b229f654 (127.5.46.65:46395)
I20260928 17:56:20.552605 5875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:51711
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:20.576294 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:20.659951 6071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:20.660182 6071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:20.660208 6071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:20.661554 6071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:20.661609 6071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:20.661639 6071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:20.663252 6071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:20.663537 6071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:20.663797 6071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:20.668938 6079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.668963 6076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.668949 6077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:20.669137 6071 server_base.cc:1034] running on GCE node
I20260928 17:56:20.669351 6071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:20.669574 6071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:20.670769 6071 hybrid_clock.cc:648] HybridClock initialized: now 1790618180670760 us; error 38 us; skew 500 ppm
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:20.673543 6071 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:20.674070 6071 webserver.cc:460] Webserver started at http://127.5.46.66:33223/ using document root <none> and password file <none>
I20260928 17:56:20.674309 6071 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:20.674382 6071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:20.674539 6071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:20.675625 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "3509decea2e0467b9effb8a72d840266"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "afa1d09f850a8db282c0eab132a1da2d"
server_key_iv: "75c0b6fea3895d434d3b84f0af48c562"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.676059 6071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "3509decea2e0467b9effb8a72d840266"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "afa1d09f850a8db282c0eab132a1da2d"
server_key_iv: "75c0b6fea3895d434d3b84f0af48c562"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.677485 6071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:20.678236 6086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.678421 6071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:20.678529 6071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "3509decea2e0467b9effb8a72d840266"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "afa1d09f850a8db282c0eab132a1da2d"
server_key_iv: "75c0b6fea3895d434d3b84f0af48c562"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.678606 6071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:20.690999 6071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:20.693756 6071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:20.693943 6071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:20.694195 6071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:20.694519 6071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:20.694577 6071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.694622 6071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:20.694648 6071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.700970 6071 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:34829
I20260928 17:56:20.701030 6199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:34829 every 8 connection(s)
I20260928 17:56:20.701362 6071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:20.701745 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6071
I20260928 17:56:20.701848 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:20.702080 5305 external_mini_cluster.cc:1428] Setting key 858bfab5af20a798a8eac09b188bf007
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 17:56:20.713199 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.702256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:42155 (local address 127.5.46.126:35639)
0928 17:56:20.702315 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.702317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.702614 (+ 297us) server_negotiation.cc:407] Connection header received
0928 17:56:20.703597 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.703599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.703612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.703675 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.704682 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.705188 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.706982 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.707193 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.707282 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.710120 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.710122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.710122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.710131 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.711606 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.712180 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.712181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.712181 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.712235 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.712484 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.712485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.712486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.712601 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.712650 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.712982 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.713072 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 17:56:20.713462 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.702314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35639 (local address 127.5.46.66:42155)
0928 17:56:20.702490 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.702533 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.703478 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.703766 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.703769 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.704258 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:20.704565 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.704572 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.705297 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.705298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.706874 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.706878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.707308 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.707309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.707487 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.708152 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:20.708164 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:20.710004 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:20.711745 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.711753 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.711783 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.712108 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.712312 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.712313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.712313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.712419 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.712657 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:20.712661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:20.712810 (+ 149us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.712923 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.713154 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
I20260928 17:56:20.714030 6200 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35639
I20260928 17:56:20.714159 6200 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:20.714396 6200 heartbeater.cc:511] Master 127.5.46.126:35639 requested a full tablet report, sending...
I20260928 17:56:20.714872 5875 ts_manager.cc:194] Registered new tserver with Master: 3509decea2e0467b9effb8a72d840266 (127.5.46.66:34829)
I20260928 17:56:20.715278 5875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:42155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:20.731299 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:20.818511 6205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:20.818735 6205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:20.818768 6205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:20.819974 6205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:20.820016 6205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:20.820043 6205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:20.821511 6205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35639
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:20.821713 6205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:20.821918 6205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:20.827055 6213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.827051 6210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:20.827111 6211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:20.827418 6205 server_base.cc:1034] running on GCE node
I20260928 17:56:20.827682 6205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:20.827971 6205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:20.829123 6205 hybrid_clock.cc:648] HybridClock initialized: now 1790618180829100 us; error 43 us; skew 500 ppm
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:20.831666 6205 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:20.832315 6205 webserver.cc:460] Webserver started at http://127.5.46.67:42477/ using document root <none> and password file <none>
I20260928 17:56:20.832579 6205 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:20.832649 6205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:20.832813 6205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:20.833904 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "ba3e0c33199f4949bff06438a7c2f8a5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "1265c3a2ba3601055031ce685c51b5f3"
server_key_iv: "14799052145b41f3d00e895062ee3302"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.834266 6205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "ba3e0c33199f4949bff06438a7c2f8a5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "1265c3a2ba3601055031ce685c51b5f3"
server_key_iv: "14799052145b41f3d00e895062ee3302"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.835628 6205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:20.836450 6220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.836621 6205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.836724 6205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "ba3e0c33199f4949bff06438a7c2f8a5"
format_stamp: "Formatted at 2026-09-28 17:56:20 on dist-test-slave-z2t3"
server_key: "1265c3a2ba3601055031ce685c51b5f3"
server_key_iv: "14799052145b41f3d00e895062ee3302"
server_key_version: "encryptionkey@0"
I20260928 17:56:20.836819 6205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:20.861966 6205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:20.863978 6205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:20.864144 6205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:20.864385 6205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:20.864693 6205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:20.864746 6205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.864799 6205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:20.864822 6205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:20.870911 6205 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:40717
I20260928 17:56:20.870955 6333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:40717 every 8 connection(s)
I20260928 17:56:20.871282 6205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:20.876703 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6205
I20260928 17:56:20.876792 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:20.876971 5305 external_mini_cluster.cc:1428] Setting key 384fe988901c2b2f7a1be442767b9fd9
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:20.881683 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.872097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:35899 (local address 127.5.46.126:35639)
0928 17:56:20.872140 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.872142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.872351 (+ 209us) server_negotiation.cc:407] Connection header received
0928 17:56:20.873341 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.873343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.873356 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.873414 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.874065 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.874574 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.875619 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.875840 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.875944 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.878543 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.878544 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.878545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.878553 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.880145 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.880634 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.880636 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.880636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.880698 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.880964 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.880965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.880965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.881104 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.881172 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.881506 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.881612 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 17:56:20.881920 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.872067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35639 (local address 127.5.46.67:35899)
0928 17:56:20.872208 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.872253 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.873204 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.873493 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.873497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.873772 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:20.873970 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.873974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.874673 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.874674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.875524 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.875526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.875960 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.875961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.876091 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.876683 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:20.876692 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:20.878436 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:20.880247 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.880255 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.880258 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.880553 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.880766 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:20.880767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:20.880768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:20.880896 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:20.881184 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:20.881190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:20.881335 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.881444 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.881640 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":68,"threads_started":1}
I20260928 17:56:20.882462 6334 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35639
I20260928 17:56:20.882596 6334 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:20.882782 6334 heartbeater.cc:511] Master 127.5.46.126:35639 requested a full tablet report, sending...
I20260928 17:56:20.883173 5875 ts_manager.cc:194] Registered new tserver with Master: ba3e0c33199f4949bff06438a7c2f8a5 (127.5.46.67:40717)
I20260928 17:56:20.883559 5875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:35899
I20260928 17:56:20.888790 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:56:20.897871 5914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.890329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.5.46.126:35639)
0928 17:56:20.890397 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.890398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.890521 (+ 123us) server_negotiation.cc:407] Connection header received
0928 17:56:20.890561 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.890561 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.890570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.890617 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.891142 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.891624 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.892425 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.892610 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.892664 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.894087 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.894089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.894089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.894097 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.896649 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.897097 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.897098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.897098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.897154 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.897396 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.897397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.897397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.897559 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.897639 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.897735 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.897802 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 17:56:20.900038 5875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:20.901625 5875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:20.908730 6349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.5.46.67:40717)
0928 17:56:20.904743 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.904747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.904757 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:20.904806 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.904808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.904821 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.905105 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:20.905186 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906333 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.907698 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908270 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908368 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908443 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.908591 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.908630 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":118,"threads_started":1}
I20260928 17:56:20.908738 6345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:40717 (local address 127.0.0.1:53908)
0928 17:56:20.904387 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.904428 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.904496 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.904919 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.904922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.904925 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:20.905003 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.905006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906488 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.906489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.907615 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.907617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908371 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.908454 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908470 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.908571 (+ 101us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.908629 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":51,"thread_start_us":78,"threads_started":1}
I20260928 17:56:20.908821 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.5.46.66:34829)
0928 17:56:20.904951 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.904955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.904965 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:20.905007 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.905009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.905024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.905110 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:20.905384 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906057 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.907480 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908291 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908364 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908428 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.908687 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.908730 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":79,"threads_started":1}
I20260928 17:56:20.908941 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60398 (local address 127.5.46.65:46395)
0928 17:56:20.904775 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.904853 (+ 78us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.904980 (+ 127us) server_negotiation.cc:407] Connection header received
0928 17:56:20.905021 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.905023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.905037 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.905109 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:20.905366 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906420 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.907664 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908376 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908433 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908478 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.908772 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.908812 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260928 17:56:20.909080 6346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:34829 (local address 127.0.0.1:56976)
0928 17:56:20.904615 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.904624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.904670 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.905200 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.905201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.905203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:20.905301 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.905303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906160 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.906162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.907387 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.907391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908464 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.908538 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908542 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.908593 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.908640 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":94,"threads_started":1}
I20260928 17:56:20.908741 6347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.904365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46395 (local address 127.0.0.1:60398)
0928 17:56:20.904728 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.904739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.904915 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.905200 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.905201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.905205 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:20.905278 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.905280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.906521 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.906522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.907572 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.907574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.908504 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.908505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.908565 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.908569 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.908614 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.908645 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":41,"thread_start_us":65,"threads_started":1}
I20260928 17:56:20.909960 6134 tablet_service.cc:1467] Processing CreateTablet for tablet 0c07450f5aa745d88806b60428e6cf5d (DEFAULT_TABLE table=test-table [id=c974529ffe564b6496d601face8e0cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:20.909966 6000 tablet_service.cc:1467] Processing CreateTablet for tablet 0c07450f5aa745d88806b60428e6cf5d (DEFAULT_TABLE table=test-table [id=c974529ffe564b6496d601face8e0cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:20.910321 6000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c07450f5aa745d88806b60428e6cf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:20.910327 6134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c07450f5aa745d88806b60428e6cf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:20.913249 6351 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: Bootstrap starting.
I20260928 17:56:20.913383 6268 tablet_service.cc:1467] Processing CreateTablet for tablet 0c07450f5aa745d88806b60428e6cf5d (DEFAULT_TABLE table=test-table [id=c974529ffe564b6496d601face8e0cb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:20.913756 6268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c07450f5aa745d88806b60428e6cf5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:20.914096 6351 tablet_bootstrap.cc:654] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:20.914654 6351 log.cc:824] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:20.915406 6351 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: No bootstrap required, opened a new log
I20260928 17:56:20.915494 6351 ts_tablet_manager.cc:1397] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:20.916277 6353 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Bootstrap starting.
I20260928 17:56:20.916776 6354 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: Bootstrap starting.
I20260928 17:56:20.917130 6353 tablet_bootstrap.cc:654] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:20.917426 6354 tablet_bootstrap.cc:654] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:20.917725 6353 log.cc:824] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:20.917598 6351 raft_consensus.cc:348] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.917804 6351 raft_consensus.cc:374] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:20.917858 6351 raft_consensus.cc:729] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5f02625de34c7bb627d4b9b229f654, State: Initialized, Role: FOLLOWER
I20260928 17:56:20.917999 6354 log.cc:824] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:20.917982 6351 consensus_queue.cc:260] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.918282 6351 ts_tablet_manager.cc:1428] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:20.918296 6066 heartbeater.cc:503] Master 127.5.46.126:35639 was elected leader, sending a full tablet report...
I20260928 17:56:20.918567 6353 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: No bootstrap required, opened a new log
I20260928 17:56:20.918656 6353 ts_tablet_manager.cc:1397] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:20.918857 6354 tablet_bootstrap.cc:492] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: No bootstrap required, opened a new log
I20260928 17:56:20.918926 6354 ts_tablet_manager.cc:1397] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:20.920876 6353 raft_consensus.cc:348] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.920876 6354 raft_consensus.cc:348] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.921108 6353 raft_consensus.cc:374] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:20.921108 6354 raft_consensus.cc:374] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:20.921149 6353 raft_consensus.cc:729] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3509decea2e0467b9effb8a72d840266, State: Initialized, Role: FOLLOWER
I20260928 17:56:20.921149 6354 raft_consensus.cc:729] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba3e0c33199f4949bff06438a7c2f8a5, State: Initialized, Role: FOLLOWER
I20260928 17:56:20.921298 6353 consensus_queue.cc:260] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.921288 6354 consensus_queue.cc:260] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.921605 6353 ts_tablet_manager.cc:1428] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:20.921605 6354 ts_tablet_manager.cc:1428] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:20.921645 6334 heartbeater.cc:503] Master 127.5.46.126:35639 was elected leader, sending a full tablet report...
I20260928 17:56:20.921727 6200 heartbeater.cc:503] Master 127.5.46.126:35639 was elected leader, sending a full tablet report...
I20260928 17:56:20.937634 6359 raft_consensus.cc:482] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:20.937768 6359 raft_consensus.cc:504] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.938092 6359 leader_election.cc:288] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3509decea2e0467b9effb8a72d840266 (127.5.46.66:34829), 6d5f02625de34c7bb627d4b9b229f654 (127.5.46.65:46395)
I20260928 17:56:20.941630 6336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.938299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46395 (local address 127.5.46.67:34795)
0928 17:56:20.938369 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.938382 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.938427 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.938645 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.938645 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.938647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:20.938717 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.938721 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.939677 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.939678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.940694 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.940697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.941355 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.941356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.941436 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.941441 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.941493 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.941540 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":37}
I20260928 17:56:20.941751 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.938420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:55821 (local address 127.5.46.66:34829)
0928 17:56:20.938481 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.938483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.938594 (+ 111us) server_negotiation.cc:407] Connection header received
0928 17:56:20.938662 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.938664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.938675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.938726 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:20.938869 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.939549 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.940779 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.941293 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.941383 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.941393 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.941600 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.941643 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 17:56:20.941782 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.938320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:34829 (local address 127.5.46.67:55821)
0928 17:56:20.938525 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:20.938533 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:20.938587 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:20.938746 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:20.938746 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:20.938748 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:20.938803 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.938805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.939623 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.939624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.940681 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:20.940684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.941422 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.941422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:20.941496 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.941498 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:56:20.941571 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 17:56:20.941601 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":87,"thread_start_us":122,"threads_started":1}
I20260928 17:56:20.941790 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.938418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34795 (local address 127.5.46.65:46395)
0928 17:56:20.938481 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.938483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.938494 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:20.938529 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.938530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.938540 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.938627 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:20.938800 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.939608 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.940796 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.941275 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.941349 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.941362 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.941675 (+ 313us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.941723 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:56:20.942104 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c07450f5aa745d88806b60428e6cf5d" candidate_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3509decea2e0467b9effb8a72d840266" is_pre_election: true
I20260928 17:56:20.942266 6154 raft_consensus.cc:2454] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3e0c33199f4949bff06438a7c2f8a5 in term 0.
I20260928 17:56:20.942368 6020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c07450f5aa745d88806b60428e6cf5d" candidate_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5f02625de34c7bb627d4b9b229f654" is_pre_election: true
I20260928 17:56:20.942510 6020 raft_consensus.cc:2454] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba3e0c33199f4949bff06438a7c2f8a5 in term 0.
I20260928 17:56:20.942518 6224 leader_election.cc:302] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3509decea2e0467b9effb8a72d840266, ba3e0c33199f4949bff06438a7c2f8a5; no voters:
I20260928 17:56:20.942638 6359 raft_consensus.cc:2781] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:20.942720 6359 raft_consensus.cc:482] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:20.942746 6359 raft_consensus.cc:3037] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:20.943486 6359 raft_consensus.cc:504] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.943634 6359 leader_election.cc:288] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [CANDIDATE]: Term 1 election: Requested vote from peers 3509decea2e0467b9effb8a72d840266 (127.5.46.66:34829), 6d5f02625de34c7bb627d4b9b229f654 (127.5.46.65:46395)
I20260928 17:56:20.943776 6154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c07450f5aa745d88806b60428e6cf5d" candidate_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3509decea2e0467b9effb8a72d840266"
I20260928 17:56:20.943776 6020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c07450f5aa745d88806b60428e6cf5d" candidate_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5f02625de34c7bb627d4b9b229f654"
I20260928 17:56:20.943874 6154 raft_consensus.cc:3037] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:20.943878 6020 raft_consensus.cc:3037] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:20.944736 6154 raft_consensus.cc:2454] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3e0c33199f4949bff06438a7c2f8a5 in term 1.
I20260928 17:56:20.944736 6020 raft_consensus.cc:2454] T 0c07450f5aa745d88806b60428e6cf5d P 6d5f02625de34c7bb627d4b9b229f654 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba3e0c33199f4949bff06438a7c2f8a5 in term 1.
I20260928 17:56:20.944919 6221 leader_election.cc:302] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5f02625de34c7bb627d4b9b229f654, ba3e0c33199f4949bff06438a7c2f8a5; no voters:
I20260928 17:56:20.945024 6359 raft_consensus.cc:2781] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:20.945176 6359 raft_consensus.cc:686] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [term 1 LEADER]: Becoming Leader. State: Replica: ba3e0c33199f4949bff06438a7c2f8a5, State: Running, Role: LEADER
I20260928 17:56:20.945278 6359 consensus_queue.cc:237] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } }
I20260928 17:56:20.946106 5874 catalog_manager.cc:5495] T 0c07450f5aa745d88806b60428e6cf5d P ba3e0c33199f4949bff06438a7c2f8a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba3e0c33199f4949bff06438a7c2f8a5 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba3e0c33199f4949bff06438a7c2f8a5" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 40717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3509decea2e0467b9effb8a72d840266" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d5f02625de34c7bb627d4b9b229f654" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46395 } health_report { overall_health: UNKNOWN } } }
W20260928 17:56:20.951959 6201 tablet.cc:2369] T 0c07450f5aa745d88806b60428e6cf5d P 3509decea2e0467b9effb8a72d840266: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:20.959157 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.952095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.5.46.65:46395)
0928 17:56:20.952144 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.952146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.952245 (+ 99us) server_negotiation.cc:407] Connection header received
0928 17:56:20.952302 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.952302 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.952311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.952358 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.952861 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.953319 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.953995 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.954174 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.954224 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.956557 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.956568 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.956568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.956575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.958062 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.958391 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.958394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.958394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.958447 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.958659 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.958659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.958660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.958791 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.958889 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.959027 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.959092 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:20.982195 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.975107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.5.46.65:46395)
0928 17:56:20.975168 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.975170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.975213 (+ 43us) server_negotiation.cc:407] Connection header received
0928 17:56:20.975294 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.975295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.975304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.975358 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.975998 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.976478 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.977183 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.977359 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:20.977407 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:20.979648 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:20.979649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:20.979650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:20.979657 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:20.980927 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.981279 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.981280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.981280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.981335 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:20.981549 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:20.981550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:20.981550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:20.981655 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:20.981812 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:20.982000 (+ 188us) server_negotiation.cc:299] Negotiation successful
0928 17:56:20.982086 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 17:56:20 dist-test-slave-z2t3 krb5kdc[5830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:21.005404 6350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:20.998269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.5.46.65:46395)
0928 17:56:20.998373 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:20.998375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:20.998386 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:20.998449 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:20.998450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:20.998459 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:20.998508 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:20.999037 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:20.999509 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:21.000328 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:21.000504 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:21.000561 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:21.003006 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:21.003007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:21.003008 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:21.003015 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:21.004492 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:21.004805 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:21.004805 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:21.004806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:21.004854 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:21.005012 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:21.005012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:21.005012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:21.005102 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:21.005169 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:21.005282 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 17:56:21.005338 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260928 17:56:21.005908 5980 tablet_service.cc:2908] Rejecting scan request for tablet 0c07450f5aa745d88806b60428e6cf5d: Uninitialized: clean time has not yet been initialized
I20260928 17:56:21.006877 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5937
I20260928 17:56:21.011749 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6071
I20260928 17:56:21.016465 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6205
I20260928 17:56:21.020726 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 5843
2026-09-28T17:56:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1983 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:21 dist-test-slave-z2t3 krb5kdc[6375](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:21 dist-test-slave-z2t3 krb5kdc[6375](info): set up 2 sockets
Sep 28 17:56:21 dist-test-slave-z2t3 krb5kdc[6375](info): commencing operation
krb5kdc: starting...
W20260928 17:56:22.059535 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:22.187206 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41615
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:22.269063 6388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:22.269309 6388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:22.269335 6388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:22.270586 6388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:22.270642 6388 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 17:56:22.270656 6388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:22.270669 6388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:22.270682 6388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:22.270694 6388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:22.272346 6388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:41615
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:22.272629 6388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:22.272882 6388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:22.279403 6388 server_base.cc:1034] running on GCE node
W20260928 17:56:22.279386 6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.279392 6396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.279605 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:22.280006 6388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:22.280279 6388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:22.281447 6388 hybrid_clock.cc:648] HybridClock initialized: now 1790618182281429 us; error 34 us; skew 500 ppm
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:22.284256 6388 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:22.284714 6388 webserver.cc:460] Webserver started at http://127.5.46.126:42131/ using document root <none> and password file <none>
I20260928 17:56:22.284993 6388 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:22.285068 6388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:22.285228 6388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:22.286396 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "34572b072b2b4dbab2a904beabbe192d"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bc91ba7bfa9c115c20b07afce73e3a54"
server_key_iv: "153101af8ba07ef88efee851a117d7c1"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.286799 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "34572b072b2b4dbab2a904beabbe192d"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bc91ba7bfa9c115c20b07afce73e3a54"
server_key_iv: "153101af8ba07ef88efee851a117d7c1"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.288391 6388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:22.289311 6403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.289525 6388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:22.289636 6388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "34572b072b2b4dbab2a904beabbe192d"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bc91ba7bfa9c115c20b07afce73e3a54"
server_key_iv: "153101af8ba07ef88efee851a117d7c1"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.289719 6388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:22.303939 6388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:22.306201 6388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:22.306404 6388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:22.310894 6388 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:41615
I20260928 17:56:22.311018 6465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:41615 every 8 connection(s)
I20260928 17:56:22.311338 6388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:22.312146 6466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.312309 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6388
I20260928 17:56:22.312415 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:22.312595 5305 external_mini_cluster.cc:1428] Setting key 96bb9051d0b63b760a9a50d6cd14107e
I20260928 17:56:22.315725 6466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Bootstrap starting.
I20260928 17:56:22.316475 6466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.316787 6466 log.cc:824] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:22.317615 6466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: No bootstrap required, opened a new log
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:22.319468 6466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } }
I20260928 17:56:22.319633 6466 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.319685 6466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34572b072b2b4dbab2a904beabbe192d, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.319831 6466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } }
I20260928 17:56:22.319967 6466 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:22.319993 6466 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:22.320063 6466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:22.320869 6466 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } }
I20260928 17:56:22.320991 6466 leader_election.cc:302] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34572b072b2b4dbab2a904beabbe192d; no voters:
I20260928 17:56:22.321171 6466 leader_election.cc:288] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:22.321347 6471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:22.321460 6466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:22.321482 6471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [term 1 LEADER]: Becoming Leader. State: Replica: 34572b072b2b4dbab2a904beabbe192d, State: Running, Role: LEADER
I20260928 17:56:22.321590 6471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } }
I20260928 17:56:22.321986 6471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34572b072b2b4dbab2a904beabbe192d. Latest consensus state: current_term: 1 leader_uuid: "34572b072b2b4dbab2a904beabbe192d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } } }
I20260928 17:56:22.322064 6471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:22.322217 6472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34572b072b2b4dbab2a904beabbe192d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34572b072b2b4dbab2a904beabbe192d" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41615 } } }
I20260928 17:56:22.322285 6472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:22.322403 6480 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:22.323002 6480 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:22.324368 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.313109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.5.46.126:41615)
0928 17:56:22.313325 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.313336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.313381 (+ 45us) server_negotiation.cc:407] Connection header received
0928 17:56:22.313901 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.313910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.313979 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.314142 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.314775 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.316199 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.317018 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.317596 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.317713 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.320196 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.320201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.320201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.320214 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.322458 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.322878 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.322881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.322881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.323002 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.323267 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.323268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.323268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.323615 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.323795 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.323992 (+ 197us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.324138 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260928 17:56:22.325103 6480 catalog_manager.cc:1347] Generated new cluster ID: 41e3ec52c9a6461a8275e0d21dad4492
I20260928 17:56:22.325158 6480 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:22.349156 6480 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:22.349952 6480 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:22.354920 6480 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Generated new TSK 0
I20260928 17:56:22.355149 6480 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:22.389343 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41615
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:22.474655 6492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:22.474895 6492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:22.474939 6492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:22.476263 6492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:22.476330 6492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:22.476367 6492 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:22.476423 6492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:22.478500 6492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:22.478790 6492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:22.479038 6492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:22.484529 6492 server_base.cc:1034] running on GCE node
W20260928 17:56:22.484535 6498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.484678 6500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.484541 6497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:22.484993 6492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:22.485234 6492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:22.486374 6492 hybrid_clock.cc:648] HybridClock initialized: now 1790618182486354 us; error 42 us; skew 500 ppm
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:22.488946 6492 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:22.489472 6492 webserver.cc:460] Webserver started at http://127.5.46.65:45333/ using document root <none> and password file <none>
I20260928 17:56:22.489704 6492 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:22.489778 6492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:22.489924 6492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:22.491001 6492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "b5a96c9efc104b268a243c4e70011e7f"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bd5de6e75c7802b55bb462e31200a4a2"
server_key_iv: "ee3aa561076affe5b2406554cf6f5caa"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.491374 6492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "b5a96c9efc104b268a243c4e70011e7f"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bd5de6e75c7802b55bb462e31200a4a2"
server_key_iv: "ee3aa561076affe5b2406554cf6f5caa"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.492832 6492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:22.493563 6507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.493729 6492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:22.493852 6492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "b5a96c9efc104b268a243c4e70011e7f"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "bd5de6e75c7802b55bb462e31200a4a2"
server_key_iv: "ee3aa561076affe5b2406554cf6f5caa"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.493935 6492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:22.519644 6492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:22.522382 6492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:22.522593 6492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:22.523123 6492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:22.523191 6492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.523228 6492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:22.523272 6492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.530143 6492 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:38259
I20260928 17:56:22.530210 6622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:38259 every 8 connection(s)
I20260928 17:56:22.530558 6492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:22.534166 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.523742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:45097 (local address 127.5.46.126:41615)
0928 17:56:22.523817 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.523820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.524053 (+ 233us) server_negotiation.cc:407] Connection header received
0928 17:56:22.524638 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.524639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.524652 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.524716 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.525704 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.526532 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.528282 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.528484 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.528544 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.531351 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.531352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.531353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.531360 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.532686 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.533136 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.533138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.533138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.533215 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.533573 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.533574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.533574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.533699 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.533781 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.534027 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.534100 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 17:56:22.534462 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.523739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.65:45097)
0928 17:56:22.523941 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.523981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.524517 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.524829 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.524835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.525265 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.525580 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.525587 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.526629 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.526630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.528175 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.528179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.528574 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.528575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.528732 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.529393 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.529407 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.531245 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.532773 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.532779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.532781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.533039 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.533328 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.533330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.533330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.533466 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.533771 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.533776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.533905 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.534001 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.534138 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":129,"threads_started":1}
I20260928 17:56:22.535030 6623 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41615
I20260928 17:56:22.535138 6623 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:22.535272 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6492
I20260928 17:56:22.535370 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:22.535408 6623 heartbeater.cc:511] Master 127.5.46.126:41615 requested a full tablet report, sending...
I20260928 17:56:22.535544 5305 external_mini_cluster.cc:1428] Setting key 9777cccd7652289f719e48c9382a8e88
I20260928 17:56:22.536056 6419 ts_manager.cc:194] Registered new tserver with Master: b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259)
I20260928 17:56:22.537005 6419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:45097
I20260928 17:56:22.544765 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.537524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.65:35303)
0928 17:56:22.537657 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.537670 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.537746 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.537906 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.537907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.538296 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.538357 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.538359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.539135 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.539140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.539717 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.539719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.540325 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.540327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.540423 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.541035 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.541040 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.541449 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.543546 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.543548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.543548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.543783 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.544044 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.544045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.544045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.544101 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.544469 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.544470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.544519 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.544589 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.544688 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260928 17:56:22.544840 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.537476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:35303 (local address 127.5.46.126:41615)
0928 17:56:22.537531 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.537533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.537754 (+ 221us) server_negotiation.cc:407] Connection header received
0928 17:56:22.537811 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.537812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.537826 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.537881 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.538470 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.539024 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.539808 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.540203 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.540287 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.541556 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.541557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.541558 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.541567 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.543431 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.543901 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.543903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.543904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.543970 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.544170 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.544172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.544172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.544339 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.544425 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.544643 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.544736 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:22.566967 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41615
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:22.649170 6632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:22.649412 6632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:22.649438 6632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:22.650681 6632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:22.650735 6632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:22.650758 6632 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:22.650782 6632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:22.652365 6632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:22.652642 6632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:22.652948 6632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:22.658239 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.658213 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.658236 6640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:22.658519 6632 server_base.cc:1034] running on GCE node
I20260928 17:56:22.658749 6632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:22.659018 6632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:22.660173 6632 hybrid_clock.cc:648] HybridClock initialized: now 1790618182660158 us; error 27 us; skew 500 ppm
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:22.662982 6632 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:22.663614 6632 webserver.cc:460] Webserver started at http://127.5.46.66:37519/ using document root <none> and password file <none>
I20260928 17:56:22.663947 6632 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:22.664006 6632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:22.664173 6632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:22.665438 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "41695cb8b9aa4ad5b0b1c450e28945df"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "ea015a6fab31903d8f9e4951995c258b"
server_key_iv: "326ffba81c126691eaafcff6c4602bff"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.665874 6632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "41695cb8b9aa4ad5b0b1c450e28945df"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "ea015a6fab31903d8f9e4951995c258b"
server_key_iv: "326ffba81c126691eaafcff6c4602bff"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.667490 6632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:22.668488 6647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.668728 6632 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:22.668838 6632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "41695cb8b9aa4ad5b0b1c450e28945df"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "ea015a6fab31903d8f9e4951995c258b"
server_key_iv: "326ffba81c126691eaafcff6c4602bff"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.668931 6632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:22.689801 6632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:22.692335 6632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:22.692513 6632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:22.693158 6632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:22.693212 6632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.693248 6632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:22.693285 6632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:22.701119 6632 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:39253
I20260928 17:56:22.701556 6632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:22.701608 6762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:39253 every 8 connection(s)
I20260928 17:56:22.702306 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6632
I20260928 17:56:22.702416 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:22.702623 5305 external_mini_cluster.cc:1428] Setting key c02b7045811bba17a5b4637bb3760fa1
I20260928 17:56:22.704290 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.693575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:51515 (local address 127.5.46.126:41615)
0928 17:56:22.693640 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.693643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.693898 (+ 255us) server_negotiation.cc:407] Connection header received
0928 17:56:22.694424 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.694426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.694439 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.694531 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.695592 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.696282 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.697671 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.697999 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.698103 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.700630 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.700631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.700632 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.700648 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.702625 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.703184 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.703185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.703185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.703255 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.703595 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.703596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.703597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.703708 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.703763 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.704158 (+ 395us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.704227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:56:22.704658 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.693584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.66:51515)
0928 17:56:22.693751 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.693793 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.694318 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.694599 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.694603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.695086 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.695481 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.695488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.696389 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.696390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.697569 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.697572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.698125 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.698126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.698275 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.698892 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.698901 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.700505 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.702756 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.702762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.702764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.703102 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.703337 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.703338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.703338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.703519 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.703779 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.703785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.704008 (+ 223us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.704143 (+ 135us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.704344 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260928 17:56:22.706048 6763 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41615
I20260928 17:56:22.706112 6763 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:22.706300 6763 heartbeater.cc:511] Master 127.5.46.126:41615 requested a full tablet report, sending...
I20260928 17:56:22.706764 6420 ts_manager.cc:194] Registered new tserver with Master: 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66:39253)
I20260928 17:56:22.707301 6420 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:51515
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 17:56:22.713112 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.706256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.66:46133)
0928 17:56:22.706434 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.706445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.706510 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.706698 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.706698 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.706927 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.707001 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.707003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.708008 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.708009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.708980 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.708984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.709409 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.709410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.709507 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.709951 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.709957 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.710425 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.712095 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.712097 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.712097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.712329 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.712527 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.712528 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.712529 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.712594 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.712830 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.712831 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.712895 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.712955 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.713045 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260928 17:56:22.713105 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.706314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:46133 (local address 127.5.46.126:41615)
0928 17:56:22.706401 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.706403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.706586 (+ 183us) server_negotiation.cc:407] Connection header received
0928 17:56:22.706609 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.706610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.706621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.706674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.707088 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.707890 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.709083 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.709322 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.709388 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.710513 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.710514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.710515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.710521 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.712014 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.712406 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.712407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.712407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.712460 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.712645 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.712646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.712646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.712775 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.712824 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.712975 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.713032 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:22.731231 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41615
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:22.815026 6772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:22.815236 6772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:22.815268 6772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:22.816473 6772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:22.816515 6772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:22.816536 6772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:22.816555 6772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:22.817984 6772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:41615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:22.818240 6772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:22.818482 6772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:22.824276 6772 server_base.cc:1034] running on GCE node
W20260928 17:56:22.824223 6780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.824210 6778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:22.824493 6777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:22.824714 6772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:22.824980 6772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:22.826082 6772 hybrid_clock.cc:648] HybridClock initialized: now 1790618182826072 us; error 32 us; skew 500 ppm
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:22.828583 6772 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:22.829031 6772 webserver.cc:460] Webserver started at http://127.5.46.67:35663/ using document root <none> and password file <none>
I20260928 17:56:22.829262 6772 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:22.829331 6772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:22.829485 6772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:22.830557 6772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "ef26144a81c24a259eda43cab1ee8c60"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "efd18907a56d90e75ce940dd63f9a6ee"
server_key_iv: "8631be4bd153f72bb5198107b773ae2f"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.830925 6772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "ef26144a81c24a259eda43cab1ee8c60"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "efd18907a56d90e75ce940dd63f9a6ee"
server_key_iv: "8631be4bd153f72bb5198107b773ae2f"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.832402 6772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:22.833211 6787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.833469 6772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:22.833575 6772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "ef26144a81c24a259eda43cab1ee8c60"
format_stamp: "Formatted at 2026-09-28 17:56:22 on dist-test-slave-z2t3"
server_key: "efd18907a56d90e75ce940dd63f9a6ee"
server_key_iv: "8631be4bd153f72bb5198107b773ae2f"
server_key_version: "encryptionkey@0"
I20260928 17:56:22.833648 6772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:22.845528 6772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:22.847537 6772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:22.847712 6772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:22.848371 6772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:22.848443 6772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:22.848484 6772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:22.848506 6772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:22.855873 6772 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:42421
I20260928 17:56:22.855929 6902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:42421 every 8 connection(s)
I20260928 17:56:22.856232 6772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:22.856576 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 6772
I20260928 17:56:22.856669 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:22.857112 5305 external_mini_cluster.cc:1428] Setting key c5fba32d8f47bacd76c36af749d38cc4
I20260928 17:56:22.858959 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.848825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:59281 (local address 127.5.46.126:41615)
0928 17:56:22.848897 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.848900 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.849131 (+ 231us) server_negotiation.cc:407] Connection header received
0928 17:56:22.849598 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.849600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.849613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.849688 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.850374 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.851180 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.852559 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.852765 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.852848 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.855870 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.855871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.855872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.855881 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.857560 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.857985 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.857986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.857987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.858046 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.858307 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.858308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.858308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.858435 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.858513 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.858790 (+ 277us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.858870 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 17:56:22.859151 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.848818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.67:59281)
0928 17:56:22.848987 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.849028 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.849502 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.849783 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.849786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.850084 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.850277 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.850281 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.851274 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.851275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.852463 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.852466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.852869 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.852870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.852997 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.853564 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.853576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.855740 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.857654 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.857661 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.857663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.857890 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.858121 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.858122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.858123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.858235 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.858504 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.858510 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.858652 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.858757 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.858924 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260928 17:56:22.859639 6903 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41615
I20260928 17:56:22.859723 6903 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:22.859948 6903 heartbeater.cc:511] Master 127.5.46.126:41615 requested a full tablet report, sending...
I20260928 17:56:22.860364 6419 ts_manager.cc:194] Registered new tserver with Master: ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421)
I20260928 17:56:22.860841 6419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:59281
I20260928 17:56:22.865912 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.860750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.5.46.67:33229)
0928 17:56:22.860908 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.860919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.860974 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.861094 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.861095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.861346 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:22.861411 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.861413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.862020 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.862021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.862511 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.862513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.862800 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.862801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.862862 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.863359 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:22.863363 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:22.863621 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:22.864960 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.864961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.864961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.865119 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.865317 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:22.865317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:22.865318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:22.865369 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:22.865663 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:22.865663 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:22.865700 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.865762 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.865837 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260928 17:56:22.865918 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.860815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:33229 (local address 127.5.46.126:41615)
0928 17:56:22.860855 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.860857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.860964 (+ 107us) server_negotiation.cc:407] Connection header received
0928 17:56:22.861020 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.861021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.861030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.861086 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.861485 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.861937 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.862575 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.862741 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.862788 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.863695 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.863695 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.863696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.863701 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.864891 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.865183 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.865183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.865184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.865231 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.865464 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.865465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.865465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.865594 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.865665 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.865784 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.865847 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:56:22.868726 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 17:56:22 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:22.900532 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.892885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.5.46.126:41615)
0928 17:56:22.892936 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.892938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.892948 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:22.892997 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.892997 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.893006 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.893062 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:22.894033 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.894799 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.895523 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.895728 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.895793 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.898301 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:22.898302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:22.898303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:22.898310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:22.899536 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.899895 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.899896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.899896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.899957 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:22.900122 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:22.900123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:22.900123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:22.900226 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:22.900325 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.900408 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.900466 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:22.902256 6419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:22.904320 6419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:22.911602 6920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42421 (local address 127.0.0.1:35086)
0928 17:56:22.907724 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.907733 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.907786 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.908543 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.908544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.908546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:22.908588 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.908589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909698 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.909699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.910568 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.910570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911247 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.911336 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911351 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.911451 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.911509 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":37,"thread_start_us":46,"threads_started":1}
I20260928 17:56:22.911610 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35086 (local address 127.5.46.67:42421)
0928 17:56:22.908347 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.908351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.908360 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:22.908393 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.908395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.908409 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.908482 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:22.908658 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909619 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.910654 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911149 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911237 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911307 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.911471 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.911510 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":91,"threads_started":1}
I20260928 17:56:22.911779 6921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:39253 (local address 127.0.0.1:41808)
0928 17:56:22.907657 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.907701 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.907773 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.908150 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.908152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.908154 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:22.908208 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.908210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909224 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.909225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.910574 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.910575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911515 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.911582 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911587 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.911616 (+ 29us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.911665 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":24,"thread_start_us":94,"threads_started":1}
I20260928 17:56:22.912165 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.5.46.66:39253)
0928 17:56:22.907909 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.907915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.907926 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:22.907969 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.907971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.907988 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.908086 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:22.908277 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909137 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.910660 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911411 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911920 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911983 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.912010 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.912056 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260928 17:56:22.912380 6922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:38259 (local address 127.0.0.1:46712)
0928 17:56:22.907896 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.907903 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.907956 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.908107 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.908110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.908112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:22.908165 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.908166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909447 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.909448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.910556 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.910558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911568 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.911625 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911627 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.912145 (+ 518us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.912199 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260928 17:56:22.912304 6837 tablet_service.cc:1467] Processing CreateTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 (DEFAULT_TABLE table=test-table [id=c59ca04024134498b356214c2322e667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.912663 6837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aecd40a02494b5b9e1b641626d9f664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.912690 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.907583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.5.46.65:38259)
0928 17:56:22.907909 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.907915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.907946 (+ 31us) server_negotiation.cc:407] Connection header received
0928 17:56:22.908007 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.908009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.908024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.908102 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:22.908208 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.909364 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.910660 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.911411 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.911485 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.911541 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.912553 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.912597 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":70,"threads_started":1}
I20260928 17:56:22.912871 6697 tablet_service.cc:1467] Processing CreateTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 (DEFAULT_TABLE table=test-table [id=c59ca04024134498b356214c2322e667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.913194 6697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aecd40a02494b5b9e1b641626d9f664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.915167 6926 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Bootstrap starting.
I20260928 17:56:22.915973 6927 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Bootstrap starting.
I20260928 17:56:22.916229 6926 tablet_bootstrap.cc:654] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.916457 6926 log.cc:824] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:22.916589 6927 tablet_bootstrap.cc:654] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.916572 6557 tablet_service.cc:1467] Processing CreateTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 (DEFAULT_TABLE table=test-table [id=c59ca04024134498b356214c2322e667]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.916894 6927 log.cc:824] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:22.916909 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aecd40a02494b5b9e1b641626d9f664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.917464 6926 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: No bootstrap required, opened a new log
I20260928 17:56:22.917546 6926 ts_tablet_manager.cc:1397] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:22.917644 6927 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: No bootstrap required, opened a new log
I20260928 17:56:22.917717 6927 ts_tablet_manager.cc:1397] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:22.919103 6926 raft_consensus.cc:348] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.919276 6926 raft_consensus.cc:374] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.919322 6926 raft_consensus.cc:729] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef26144a81c24a259eda43cab1ee8c60, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.919430 6926 consensus_queue.cc:260] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.919700 6926 ts_tablet_manager.cc:1428] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:22.919729 6903 heartbeater.cc:503] Master 127.5.46.126:41615 was elected leader, sending a full tablet report...
I20260928 17:56:22.920037 6927 raft_consensus.cc:348] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.920223 6927 raft_consensus.cc:374] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.920264 6927 raft_consensus.cc:729] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41695cb8b9aa4ad5b0b1c450e28945df, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.920398 6927 consensus_queue.cc:260] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.920651 6927 ts_tablet_manager.cc:1428] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:22.920800 6763 heartbeater.cc:503] Master 127.5.46.126:41615 was elected leader, sending a full tablet report...
I20260928 17:56:22.920907 6932 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Bootstrap starting.
I20260928 17:56:22.921547 6932 tablet_bootstrap.cc:654] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.921849 6932 log.cc:824] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:22.922689 6932 tablet_bootstrap.cc:492] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: No bootstrap required, opened a new log
I20260928 17:56:22.922768 6932 ts_tablet_manager.cc:1397] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:22.924410 6932 raft_consensus.cc:348] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.924563 6932 raft_consensus.cc:374] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.924609 6932 raft_consensus.cc:729] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a96c9efc104b268a243c4e70011e7f, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.924728 6932 consensus_queue.cc:260] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.924975 6932 ts_tablet_manager.cc:1428] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:56:22.925043 6623 heartbeater.cc:503] Master 127.5.46.126:41615 was elected leader, sending a full tablet report...
I20260928 17:56:22.930658 6931 raft_consensus.cc:482] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:22.930747 6931 raft_consensus.cc:504] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.931064 6931 leader_election.cc:288] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421), b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259)
I20260928 17:56:22.934984 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.931331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:59875 (local address 127.5.46.65:38259)
0928 17:56:22.931371 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.931373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.931554 (+ 181us) server_negotiation.cc:407] Connection header received
0928 17:56:22.931594 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.931596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.931606 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.931654 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:22.931803 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.932529 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934028 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.934591 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934659 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.934669 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.934885 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.934921 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:56:22.934984 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.931361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:59703 (local address 127.5.46.67:42421)
0928 17:56:22.931405 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.931407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.931417 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:22.931443 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.931444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.931454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.931511 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:22.931704 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.932511 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934028 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.934581 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934677 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.934686 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.934876 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.934916 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:22.934994 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.931262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42421 (local address 127.5.46.66:59703)
0928 17:56:22.931320 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.931332 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.931375 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.931544 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.931545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.931546 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:22.931613 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.931615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.932602 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.932603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.933931 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.933934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.934670 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.934780 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.934784 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.934849 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.934890 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":115}
I20260928 17:56:22.934999 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.931270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:38259 (local address 127.5.46.66:59875)
0928 17:56:22.931463 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:22.931471 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:22.931545 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:22.931671 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:22.931672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:22.931674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:22.931737 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.931738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.932602 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.932603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.933932 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:22.933934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.934670 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.934670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:22.934816 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.934819 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:22.934870 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 17:56:22.934898 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":92,"thread_start_us":96,"threads_started":1}
I20260928 17:56:22.935336 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aecd40a02494b5b9e1b641626d9f664" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f" is_pre_election: true
I20260928 17:56:22.935340 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aecd40a02494b5b9e1b641626d9f664" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef26144a81c24a259eda43cab1ee8c60" is_pre_election: true
I20260928 17:56:22.935473 6577 raft_consensus.cc:2454] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 0.
I20260928 17:56:22.935473 6857 raft_consensus.cc:2454] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 0.
I20260928 17:56:22.935688 6651 leader_election.cc:302] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41695cb8b9aa4ad5b0b1c450e28945df, b5a96c9efc104b268a243c4e70011e7f; no voters:
I20260928 17:56:22.936067 6931 raft_consensus.cc:2781] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:22.936144 6931 raft_consensus.cc:482] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:22.936167 6931 raft_consensus.cc:3037] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:22.936939 6931 raft_consensus.cc:504] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.937081 6931 leader_election.cc:288] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 election: Requested vote from peers ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421), b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259)
I20260928 17:56:22.937228 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aecd40a02494b5b9e1b641626d9f664" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f"
I20260928 17:56:22.937237 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aecd40a02494b5b9e1b641626d9f664" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef26144a81c24a259eda43cab1ee8c60"
I20260928 17:56:22.937296 6577 raft_consensus.cc:3037] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:22.937371 6857 raft_consensus.cc:3037] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:22.938064 6577 raft_consensus.cc:2454] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 1.
I20260928 17:56:22.938079 6857 raft_consensus.cc:2454] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 1.
I20260928 17:56:22.938221 6651 leader_election.cc:302] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41695cb8b9aa4ad5b0b1c450e28945df, b5a96c9efc104b268a243c4e70011e7f; no voters:
I20260928 17:56:22.938303 6931 raft_consensus.cc:2781] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:22.938448 6931 raft_consensus.cc:686] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 LEADER]: Becoming Leader. State: Replica: 41695cb8b9aa4ad5b0b1c450e28945df, State: Running, Role: LEADER
I20260928 17:56:22.938551 6931 consensus_queue.cc:237] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } }
I20260928 17:56:22.939388 6419 catalog_manager.cc:5495] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df reported cstate change: term changed from 0 to 1, leader changed from <none> to 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66). New cstate: current_term: 1 leader_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:22.948993 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:22.945856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.5.46.66:39253)
0928 17:56:22.945915 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:22.945918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:22.945928 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:22.946052 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:22.946053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:22.946074 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:22.946149 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:22.946382 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.946906 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.947788 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:22.948056 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:22.948161 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:22.948395 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:22.948506 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:22.948604 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:22.948664 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 17:56:22.948695 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 17:56:22.950835 6577 raft_consensus.cc:1261] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Refusing update from remote peer 41695cb8b9aa4ad5b0b1c450e28945df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:22.950838 6857 raft_consensus.cc:1261] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Refusing update from remote peer 41695cb8b9aa4ad5b0b1c450e28945df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:22.951229 6936 consensus_queue.cc:1035] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:22.951370 6931 consensus_queue.cc:1035] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 17:56:22.953161 6764 tablet.cc:2369] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:22.953738 6939 mvcc.cc:205] Tried to move back new op lower bound from 7334372077363638272 to 7334372077317074944. Current Snapshot: MvccSnapshot[applied={T|T < 7334372077363638272}]
I20260928 17:56:22.953900 6940 mvcc.cc:205] Tried to move back new op lower bound from 7334372077363638272 to 7334372077317074944. Current Snapshot: MvccSnapshot[applied={T|T < 7334372077363638272}]
I20260928 17:56:22.957232 6419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:22.957343 6419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:22.958071 6419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Sending DeleteTablet for 3 replicas of tablet 8aecd40a02494b5b9e1b641626d9f664
I20260928 17:56:22.958400 6557 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46712
I20260928 17:56:22.958446 6837 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35086
I20260928 17:56:22.958621 6697 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aecd40a02494b5b9e1b641626d9f664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41808
I20260928 17:56:22.958673 6950 tablet_replica.cc:330] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: stopping tablet replica
I20260928 17:56:22.958853 6949 tablet_replica.cc:330] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: stopping tablet replica
I20260928 17:56:22.958858 6950 raft_consensus.cc:2229] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:22.958954 6949 raft_consensus.cc:2229] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:22.958895 6419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:22.959033 6419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:22.959064 6949 raft_consensus.cc:2258] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:22.959254 6950 raft_consensus.cc:2258] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:22.959324 6951 tablet_replica.cc:330] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: stopping tablet replica
I20260928 17:56:22.959403 6951 raft_consensus.cc:2229] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:22.959901 6951 raft_consensus.cc:2258] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:22.959910 6949 ts_tablet_manager.cc:1905] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:22.960309 6557 tablet_service.cc:1467] Processing CreateTablet for tablet 1f86b12cb01649c48be00f4c6565c331 (DEFAULT_TABLE table=test-table [id=459233064b5e499f8605d1330f50a749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.960361 6697 tablet_service.cc:1467] Processing CreateTablet for tablet 1f86b12cb01649c48be00f4c6565c331 (DEFAULT_TABLE table=test-table [id=459233064b5e499f8605d1330f50a749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.960444 6697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f86b12cb01649c48be00f4c6565c331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.960443 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f86b12cb01649c48be00f4c6565c331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.961943 6927 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Bootstrap starting.
I20260928 17:56:22.962390 6927 tablet_bootstrap.cc:654] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.962430 6949 ts_tablet_manager.cc:1918] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:22.962494 6949 log.cc:1196] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/8aecd40a02494b5b9e1b641626d9f664
I20260928 17:56:22.962774 6932 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Bootstrap starting.
I20260928 17:56:22.962919 6949 ts_tablet_manager.cc:1939] T 8aecd40a02494b5b9e1b641626d9f664 P b5a96c9efc104b268a243c4e70011e7f: Deleting consensus metadata
I20260928 17:56:22.963092 6950 ts_tablet_manager.cc:1905] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:22.963160 6932 tablet_bootstrap.cc:654] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.963313 6407 catalog_manager.cc:4841] TS b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259): tablet 8aecd40a02494b5b9e1b641626d9f664 (table test-table [id=c59ca04024134498b356214c2322e667]) successfully deleted
I20260928 17:56:22.963728 6927 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: No bootstrap required, opened a new log
I20260928 17:56:22.963792 6927 ts_tablet_manager.cc:1397] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:22.963939 6837 tablet_service.cc:1467] Processing CreateTablet for tablet 1f86b12cb01649c48be00f4c6565c331 (DEFAULT_TABLE table=test-table [id=459233064b5e499f8605d1330f50a749]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:22.963953 6927 raft_consensus.cc:348] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.964033 6927 raft_consensus.cc:374] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.964059 6837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f86b12cb01649c48be00f4c6565c331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:22.964066 6927 raft_consensus.cc:729] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41695cb8b9aa4ad5b0b1c450e28945df, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.964109 6927 consensus_queue.cc:260] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.964198 6927 ts_tablet_manager.cc:1428] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:22.965219 6950 ts_tablet_manager.cc:1918] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:22.965265 6926 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Bootstrap starting.
I20260928 17:56:22.965456 6950 log.cc:1196] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/8aecd40a02494b5b9e1b641626d9f664
I20260928 17:56:22.965696 6926 tablet_bootstrap.cc:654] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:22.965683 6951 ts_tablet_manager.cc:1905] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:22.966061 6950 ts_tablet_manager.cc:1939] T 8aecd40a02494b5b9e1b641626d9f664 P ef26144a81c24a259eda43cab1ee8c60: Deleting consensus metadata
I20260928 17:56:22.966455 6405 catalog_manager.cc:4841] TS ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421): tablet 8aecd40a02494b5b9e1b641626d9f664 (table test-table [id=c59ca04024134498b356214c2322e667]) successfully deleted
I20260928 17:56:22.967252 6951 ts_tablet_manager.cc:1918] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:22.967321 6951 log.cc:1196] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/8aecd40a02494b5b9e1b641626d9f664
I20260928 17:56:22.967506 6932 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: No bootstrap required, opened a new log
I20260928 17:56:22.967578 6951 ts_tablet_manager.cc:1939] T 8aecd40a02494b5b9e1b641626d9f664 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting consensus metadata
I20260928 17:56:22.967566 6932 ts_tablet_manager.cc:1397] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 17:56:22.967741 6932 raft_consensus.cc:348] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.967821 6932 raft_consensus.cc:374] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.967861 6932 raft_consensus.cc:729] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a96c9efc104b268a243c4e70011e7f, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.967909 6932 consensus_queue.cc:260] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.967986 6404 catalog_manager.cc:4841] TS 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66:39253): tablet 8aecd40a02494b5b9e1b641626d9f664 (table test-table [id=c59ca04024134498b356214c2322e667]) successfully deleted
I20260928 17:56:22.968027 6932 ts_tablet_manager.cc:1428] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:22.976403 6926 tablet_bootstrap.cc:492] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: No bootstrap required, opened a new log
I20260928 17:56:22.976459 6926 ts_tablet_manager.cc:1397] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260928 17:56:22.976631 6926 raft_consensus.cc:348] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.976696 6926 raft_consensus.cc:374] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:22.976708 6926 raft_consensus.cc:729] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef26144a81c24a259eda43cab1ee8c60, State: Initialized, Role: FOLLOWER
I20260928 17:56:22.976763 6926 consensus_queue.cc:260] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:22.976854 6926 ts_tablet_manager.cc:1428] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260928 17:56:23.031464 6624 tablet.cc:2369] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:56:23.106945 6904 tablet.cc:2369] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:23.141002 6944 raft_consensus.cc:482] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:23.141121 6944 raft_consensus.cc:504] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.141345 6944 leader_election.cc:288] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259), ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421)
I20260928 17:56:23.141625 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f86b12cb01649c48be00f4c6565c331" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f" is_pre_election: true
I20260928 17:56:23.141753 6577 raft_consensus.cc:2454] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 0.
I20260928 17:56:23.141778 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f86b12cb01649c48be00f4c6565c331" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef26144a81c24a259eda43cab1ee8c60" is_pre_election: true
I20260928 17:56:23.141904 6857 raft_consensus.cc:2454] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 0.
I20260928 17:56:23.141952 6651 leader_election.cc:302] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41695cb8b9aa4ad5b0b1c450e28945df, b5a96c9efc104b268a243c4e70011e7f; no voters:
I20260928 17:56:23.142047 6944 raft_consensus.cc:2781] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:23.142105 6944 raft_consensus.cc:482] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:23.142138 6944 raft_consensus.cc:3037] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.142880 6944 raft_consensus.cc:504] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.143033 6944 leader_election.cc:288] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 election: Requested vote from peers b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259), ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421)
I20260928 17:56:23.143213 6857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f86b12cb01649c48be00f4c6565c331" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef26144a81c24a259eda43cab1ee8c60"
I20260928 17:56:23.143218 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f86b12cb01649c48be00f4c6565c331" candidate_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f"
I20260928 17:56:23.143301 6857 raft_consensus.cc:3037] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.143301 6577 raft_consensus.cc:3037] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.144112 6577 raft_consensus.cc:2454] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 1.
I20260928 17:56:23.144111 6857 raft_consensus.cc:2454] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41695cb8b9aa4ad5b0b1c450e28945df in term 1.
I20260928 17:56:23.144351 6649 leader_election.cc:302] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41695cb8b9aa4ad5b0b1c450e28945df, ef26144a81c24a259eda43cab1ee8c60; no voters:
I20260928 17:56:23.144464 6944 raft_consensus.cc:2781] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:23.144510 6944 raft_consensus.cc:686] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 LEADER]: Becoming Leader. State: Replica: 41695cb8b9aa4ad5b0b1c450e28945df, State: Running, Role: LEADER
I20260928 17:56:23.144567 6944 consensus_queue.cc:237] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.145272 6419 catalog_manager.cc:5495] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df reported cstate change: term changed from 0 to 1, leader changed from <none> to 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66). New cstate: current_term: 1 leader_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } health_report { overall_health: UNKNOWN } } }
Sep 28 17:56:23 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:23.172183 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.164972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32836 (local address 127.5.46.126:41615)
0928 17:56:23.165073 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.165076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.165192 (+ 116us) server_negotiation.cc:407] Connection header received
0928 17:56:23.165287 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.165287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.165298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.165374 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:23.165878 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.166436 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.167154 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.167340 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.167425 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.169648 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:23.169650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:23.169651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:23.169657 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:23.170915 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.171331 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.171332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.171332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.171390 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.171583 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.171583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.171584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.171700 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:23.171772 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.171964 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.172066 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":72}
I20260928 17:56:23.172242 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.164883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41615 (local address 127.0.0.1:32836)
0928 17:56:23.165102 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.165121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.165214 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.165381 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.165382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.165740 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:23.165793 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.165795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.166526 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.166527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.167063 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.167065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.167425 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.167426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.167483 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.168131 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:23.168136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:23.169544 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:23.171013 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:23.171021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:23.171024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:23.171217 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:23.171456 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:23.171457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:23.171458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:23.171516 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:23.171797 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:23.171803 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:23.171867 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.171941 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.172117 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":76,"thread_start_us":129,"threads_started":1}
I20260928 17:56:23.174360 6419 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:32836:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 17:56:23.174541 6419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:23.175979 6557 tablet_service.cc:1467] Processing CreateTablet for tablet 13d09ceb535648dcb70ed4aafd581a5f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07915bba8e664c9abecad16037b2579a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:23.175987 6697 tablet_service.cc:1467] Processing CreateTablet for tablet 13d09ceb535648dcb70ed4aafd581a5f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07915bba8e664c9abecad16037b2579a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:23.176012 6837 tablet_service.cc:1467] Processing CreateTablet for tablet 13d09ceb535648dcb70ed4aafd581a5f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=07915bba8e664c9abecad16037b2579a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:23.176144 6837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d09ceb535648dcb70ed4aafd581a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:23.176146 6557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d09ceb535648dcb70ed4aafd581a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:23.176144 6697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13d09ceb535648dcb70ed4aafd581a5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:23.177309 6927 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: Bootstrap starting.
I20260928 17:56:23.177640 6932 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: Bootstrap starting.
I20260928 17:56:23.177685 6927 tablet_bootstrap.cc:654] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:23.178025 6926 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: Bootstrap starting.
I20260928 17:56:23.178046 6932 tablet_bootstrap.cc:654] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:23.178427 6927 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: No bootstrap required, opened a new log
I20260928 17:56:23.178474 6927 ts_tablet_manager.cc:1397] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:23.178577 6926 tablet_bootstrap.cc:654] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:23.178586 6927 raft_consensus.cc:348] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.178635 6932 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: No bootstrap required, opened a new log
I20260928 17:56:23.178659 6927 raft_consensus.cc:374] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:23.178669 6932 ts_tablet_manager.cc:1397] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:23.178686 6927 raft_consensus.cc:729] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41695cb8b9aa4ad5b0b1c450e28945df, State: Initialized, Role: FOLLOWER
I20260928 17:56:23.178740 6927 consensus_queue.cc:260] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.178777 6932 raft_consensus.cc:348] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.178822 6932 raft_consensus.cc:374] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:23.178835 6932 raft_consensus.cc:729] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5a96c9efc104b268a243c4e70011e7f, State: Initialized, Role: FOLLOWER
I20260928 17:56:23.178846 6927 ts_tablet_manager.cc:1428] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:23.178895 6932 consensus_queue.cc:260] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.178936 6944 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.178984 6932 ts_tablet_manager.cc:1428] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:23.179010 6944 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.179092 6934 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.179169 6934 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.179585 6926 tablet_bootstrap.cc:492] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: No bootstrap required, opened a new log
I20260928 17:56:23.179638 6926 ts_tablet_manager.cc:1397] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:23.179792 6926 raft_consensus.cc:348] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.179903 6926 raft_consensus.cc:374] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:23.179924 6926 raft_consensus.cc:729] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef26144a81c24a259eda43cab1ee8c60, State: Initialized, Role: FOLLOWER
I20260928 17:56:23.180008 6926 consensus_queue.cc:260] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.180120 6926 ts_tablet_manager.cc:1428] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:23.180173 6930 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.180255 6930 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.327627 6930 raft_consensus.cc:482] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:23.327764 6930 raft_consensus.cc:504] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.328198 6930 leader_election.cc:288] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259), 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66:39253)
I20260928 17:56:23.331887 6965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.328537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:38259 (local address 127.5.46.67:37697)
0928 17:56:23.328707 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.328716 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.328781 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.328929 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.328930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.328931 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:23.329024 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.329025 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.330050 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.330051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.330944 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.330947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.331566 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.331654 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.331658 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.331714 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.331747 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":20,"thread_start_us":100,"threads_started":1}
I20260928 17:56:23.331918 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.328604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:37697 (local address 127.5.46.65:38259)
0928 17:56:23.328695 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.328697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.328803 (+ 106us) server_negotiation.cc:407] Connection header received
0928 17:56:23.328832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.328833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.328841 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.328906 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:23.329153 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.329913 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331055 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.331473 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331545 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.331555 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.331749 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.331803 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 17:56:23.331984 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.328616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:35877 (local address 127.5.46.66:39253)
0928 17:56:23.328677 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.328679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.328691 (+ 12us) server_negotiation.cc:407] Connection header received
0928 17:56:23.328724 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.328725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.328737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.328799 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:23.328975 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.329517 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331099 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.331496 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331598 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.331608 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.331835 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.331910 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:23.332010 6796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.328512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:39253 (local address 127.5.46.67:35877)
0928 17:56:23.328587 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.328599 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.328647 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.328818 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.328819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.328821 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:23.328899 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.328901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.329613 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.329615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.330949 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.330952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.331617 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.331618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.331722 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.331726 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.331790 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.331831 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":3}
I20260928 17:56:23.332228 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d09ceb535648dcb70ed4aafd581a5f" candidate_uuid: "ef26144a81c24a259eda43cab1ee8c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f" is_pre_election: true
I20260928 17:56:23.332317 6577 raft_consensus.cc:2454] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef26144a81c24a259eda43cab1ee8c60 in term 0.
I20260928 17:56:23.332434 6717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d09ceb535648dcb70ed4aafd581a5f" candidate_uuid: "ef26144a81c24a259eda43cab1ee8c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" is_pre_election: true
I20260928 17:56:23.332530 6791 leader_election.cc:302] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a96c9efc104b268a243c4e70011e7f, ef26144a81c24a259eda43cab1ee8c60; no voters:
I20260928 17:56:23.332608 6717 raft_consensus.cc:2454] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef26144a81c24a259eda43cab1ee8c60 in term 0.
I20260928 17:56:23.332660 6930 raft_consensus.cc:2781] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:23.332700 6930 raft_consensus.cc:482] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:23.332724 6930 raft_consensus.cc:3037] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.333474 6930 raft_consensus.cc:504] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.333639 6930 leader_election.cc:288] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [CANDIDATE]: Term 1 election: Requested vote from peers b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259), 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66:39253)
I20260928 17:56:23.333786 6577 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d09ceb535648dcb70ed4aafd581a5f" candidate_uuid: "ef26144a81c24a259eda43cab1ee8c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5a96c9efc104b268a243c4e70011e7f"
I20260928 17:56:23.333873 6577 raft_consensus.cc:3037] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.333850 6717 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13d09ceb535648dcb70ed4aafd581a5f" candidate_uuid: "ef26144a81c24a259eda43cab1ee8c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41695cb8b9aa4ad5b0b1c450e28945df"
I20260928 17:56:23.333920 6717 raft_consensus.cc:3037] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:23.334687 6577 raft_consensus.cc:2454] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef26144a81c24a259eda43cab1ee8c60 in term 1.
I20260928 17:56:23.334786 6717 raft_consensus.cc:2454] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef26144a81c24a259eda43cab1ee8c60 in term 1.
I20260928 17:56:23.334895 6791 leader_election.cc:302] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5a96c9efc104b268a243c4e70011e7f, ef26144a81c24a259eda43cab1ee8c60; no voters:
I20260928 17:56:23.334992 6930 raft_consensus.cc:2781] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:23.335120 6930 raft_consensus.cc:686] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [term 1 LEADER]: Becoming Leader. State: Replica: ef26144a81c24a259eda43cab1ee8c60, State: Running, Role: LEADER
I20260928 17:56:23.335222 6930 consensus_queue.cc:237] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } }
I20260928 17:56:23.335618 6966 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: TxnStatusTablet state changed. Reason: New leader ef26144a81c24a259eda43cab1ee8c60. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.335785 6966 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:23.336071 6968 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 17:56:23.336228 6968 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 17:56:23.336133 6420 catalog_manager.cc:5495] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:23.365571 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.362915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.5.46.67:42421)
0928 17:56:23.362975 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.362977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.362986 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:23.363119 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.363124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.363151 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.363233 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:23.363372 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.363954 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.364724 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.364927 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.365008 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.365227 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.365329 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.365413 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.365458 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.365485 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 17:56:23.365579 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.362856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42421 (local address 127.0.0.1:35092)
0928 17:56:23.362937 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.362949 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.363004 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.363239 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.363240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.363241 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:23.363293 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.363297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.364045 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.364047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.364629 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.364630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.365007 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.365009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.365084 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.365100 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.365397 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.365407 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.365455 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.365509 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 17:56:23.366887 6577 raft_consensus.cc:1261] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Refusing update from remote peer ef26144a81c24a259eda43cab1ee8c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:23.366994 6717 raft_consensus.cc:1261] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Refusing update from remote peer ef26144a81c24a259eda43cab1ee8c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:23.367147 6930 consensus_queue.cc:1035] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:23.367287 6966 consensus_queue.cc:1035] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:23.367709 6934 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: TxnStatusTablet state changed. Reason: New leader ef26144a81c24a259eda43cab1ee8c60. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.367803 6934 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.367962 6944 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: TxnStatusTablet state changed. Reason: New leader ef26144a81c24a259eda43cab1ee8c60. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.368058 6944 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.368643 6966 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.368733 6966 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:23.368753 6941 mvcc.cc:205] Tried to move back new op lower bound from 7334372079068483584 to 7334372078941765632. Current Snapshot: MvccSnapshot[applied={T|T < 7334372079068483584}]
I20260928 17:56:23.368845 6934 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.368911 6934 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P b5a96c9efc104b268a243c4e70011e7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.369951 6930 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.370041 6930 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P ef26144a81c24a259eda43cab1ee8c60: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:23.370442 6944 tablet_replica.cc:439] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef26144a81c24a259eda43cab1ee8c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 } } peers { permanent_uuid: "41695cb8b9aa4ad5b0b1c450e28945df" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 39253 } } peers { permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 } } }
I20260928 17:56:23.370545 6944 tablet_replica.cc:442] T 13d09ceb535648dcb70ed4aafd581a5f P 41695cb8b9aa4ad5b0b1c450e28945df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:23.376012 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.373081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:43373 (local address 127.5.46.67:42421)
0928 17:56:23.373129 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.373131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.373292 (+ 161us) server_negotiation.cc:407] Connection header received
0928 17:56:23.373348 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.373349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.373359 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.373406 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:23.373536 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.374022 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.375191 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.375428 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.375498 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.375690 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.375717 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.375762 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.375908 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.375932 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:23.376019 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.373023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42421 (local address 127.5.46.66:43373)
0928 17:56:23.373211 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.373224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.373288 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.373423 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.373424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.373426 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:23.373480 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.373482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.374097 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.374098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.375078 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.375081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.375510 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.375511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.375585 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.375605 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.375783 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.375785 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.375840 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.375923 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260928 17:56:23.378515 6577 raft_consensus.cc:1261] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Refusing update from remote peer 41695cb8b9aa4ad5b0b1c450e28945df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:23.378516 6857 raft_consensus.cc:1261] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Refusing update from remote peer 41695cb8b9aa4ad5b0b1c450e28945df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:23.378799 6931 consensus_queue.cc:1035] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef26144a81c24a259eda43cab1ee8c60" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:23.378934 6944 consensus_queue.cc:1035] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5a96c9efc104b268a243c4e70011e7f" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 38259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:23.391206 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.385607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:39253 (local address 127.5.46.67:45969)
0928 17:56:23.388056 (+ 2449us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:23.388068 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:23.388156 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:23.388373 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:23.388374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:23.388376 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:23.388455 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.388457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.389201 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.389202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.390260 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:23.390263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.390674 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.390675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:23.390766 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.390784 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.391000 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.391003 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:23.391062 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 17:56:23.391110 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2390,"thread_start_us":93,"threads_started":1}
I20260928 17:56:23.391187 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.385670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:45969 (local address 127.5.46.66:39253)
0928 17:56:23.385717 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.385719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.388142 (+ 2423us) server_negotiation.cc:407] Connection header received
0928 17:56:23.388269 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.388270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.388285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.388369 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:23.388540 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.389106 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.390346 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.390580 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.390666 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.390877 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:23.390919 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:23.390999 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.391067 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.391095 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:23.401640 6420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:23.401769 6420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:23.402511 6420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 34572b072b2b4dbab2a904beabbe192d: Sending DeleteTablet for 3 replicas of tablet 1f86b12cb01649c48be00f4c6565c331
I20260928 17:56:23.402769 6557 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f86b12cb01649c48be00f4c6565c331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:23 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46712
I20260928 17:56:23.402846 6697 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f86b12cb01649c48be00f4c6565c331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:23 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41808
I20260928 17:56:23.402953 6951 tablet_replica.cc:330] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: stopping tablet replica
I20260928 17:56:23.402961 6837 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f86b12cb01649c48be00f4c6565c331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:23 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35086
I20260928 17:56:23.403024 6951 raft_consensus.cc:2229] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:23.403093 6951 raft_consensus.cc:2258] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:23.403172 6950 tablet_replica.cc:330] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: stopping tablet replica
I20260928 17:56:23.403259 6950 raft_consensus.cc:2229] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:23.403308 6950 raft_consensus.cc:2258] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:23.403474 6949 tablet_replica.cc:330] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: stopping tablet replica
I20260928 17:56:23.403546 6949 raft_consensus.cc:2229] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:23.403594 6949 raft_consensus.cc:2258] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:23.403602 6950 ts_tablet_manager.cc:1905] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:23.403913 6951 ts_tablet_manager.cc:1905] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:23.403956 6949 ts_tablet_manager.cc:1905] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:23.405398 6950 ts_tablet_manager.cc:1918] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:23.405454 6950 log.cc:1196] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/1f86b12cb01649c48be00f4c6565c331
I20260928 17:56:23.405611 6949 ts_tablet_manager.cc:1918] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:23.405611 6951 ts_tablet_manager.cc:1918] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:23.405668 6949 log.cc:1196] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/1f86b12cb01649c48be00f4c6565c331
I20260928 17:56:23.405668 6950 ts_tablet_manager.cc:1939] T 1f86b12cb01649c48be00f4c6565c331 P ef26144a81c24a259eda43cab1ee8c60: Deleting consensus metadata
I20260928 17:56:23.405683 6951 log.cc:1196] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/1f86b12cb01649c48be00f4c6565c331
I20260928 17:56:23.405905 6951 ts_tablet_manager.cc:1939] T 1f86b12cb01649c48be00f4c6565c331 P 41695cb8b9aa4ad5b0b1c450e28945df: Deleting consensus metadata
I20260928 17:56:23.405917 6949 ts_tablet_manager.cc:1939] T 1f86b12cb01649c48be00f4c6565c331 P b5a96c9efc104b268a243c4e70011e7f: Deleting consensus metadata
I20260928 17:56:23.406040 6405 catalog_manager.cc:4841] TS ef26144a81c24a259eda43cab1ee8c60 (127.5.46.67:42421): tablet 1f86b12cb01649c48be00f4c6565c331 (table test-table [id=459233064b5e499f8605d1330f50a749]) successfully deleted
I20260928 17:56:23.406252 6404 catalog_manager.cc:4841] TS 41695cb8b9aa4ad5b0b1c450e28945df (127.5.46.66:39253): tablet 1f86b12cb01649c48be00f4c6565c331 (table test-table [id=459233064b5e499f8605d1330f50a749]) successfully deleted
I20260928 17:56:23.406279 6407 catalog_manager.cc:4841] TS b5a96c9efc104b268a243c4e70011e7f (127.5.46.65:38259): tablet 1f86b12cb01649c48be00f4c6565c331 (table test-table [id=459233064b5e499f8605d1330f50a749]) successfully deleted
Sep 28 17:56:23 dist-test-slave-z2t3 krb5kdc[6375](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618182, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:23.421960 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.403938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.5.46.65:38259)
0928 17:56:23.403991 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.403993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.408073 (+ 4080us) server_negotiation.cc:407] Connection header received
0928 17:56:23.408138 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.408139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.408149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.408195 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:23.408878 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.409380 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.410363 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.410557 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.410614 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.418632 (+ 8018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:23.418645 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:23.418645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:23.418657 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:23.420549 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.420960 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.420966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.420967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.421030 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.421288 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.421289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.421290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.421495 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:23.421653 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.421744 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.421833 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 17:56:23.422262 6617 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46728
I20260928 17:56:23.430421 6469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:23.423384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.5.46.126:41615)
0928 17:56:23.423429 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:23.423431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:23.423446 (+ 15us) server_negotiation.cc:407] Connection header received
0928 17:56:23.423530 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:23.423530 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:23.423539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:23.423576 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:23.424278 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.424810 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.425842 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:23.426035 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:23.426092 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:23.427607 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:23.427608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:23.427609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:23.427617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:23.429226 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.429629 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.429630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.429630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.429682 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:23.429904 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:23.429905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:23.429905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:23.430033 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:23.430108 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:23.430279 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 17:56:23.430351 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 17:56:23.430665 6420 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32850
I20260928 17:56:23.431499 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6492
I20260928 17:56:23.437714 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6632
I20260928 17:56:23.444023 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6772
I20260928 17:56:23.449417 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 6388
2026-09-28T17:56:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2428 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:23 dist-test-slave-z2t3 krb5kdc[6988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:23 dist-test-slave-z2t3 krb5kdc[6988](info): set up 2 sockets
Sep 28 17:56:23 dist-test-slave-z2t3 krb5kdc[6988](info): commencing operation
krb5kdc: starting...
W20260928 17:56:24.539758 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:24 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:24.658757 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:40707
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:40707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:24.739737 7001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:24.740077 7001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:24.740118 7001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:24.741323 7001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:24.741400 7001 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 17:56:24.741421 7001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:24.741442 7001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:24.741474 7001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:24.741499 7001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:24.743083 7001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:40707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:40707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:24.743387 7001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:24.743625 7001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:24.748835 7009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:24.748950 7001 server_base.cc:1034] running on GCE node
W20260928 17:56:24.748848 7006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:24.749043 7007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:24.749313 7001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:24.749626 7001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:24.750819 7001 hybrid_clock.cc:648] HybridClock initialized: now 1790618184750802 us; error 48 us; skew 500 ppm
Sep 28 17:56:24 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:24.766110 7001 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:24.766628 7001 webserver.cc:460] Webserver started at http://127.5.46.126:45095/ using document root <none> and password file <none>
I20260928 17:56:24.766847 7001 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:24.766892 7001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:24.766989 7001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:24.768152 7001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "7cb7a70b24b44613bc9a85d87a878b13"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "711cd1b441ae7247635b77768407068e"
server_key_iv: "362303354899e4d71a5accf22666225e"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.768472 7001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "7cb7a70b24b44613bc9a85d87a878b13"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "711cd1b441ae7247635b77768407068e"
server_key_iv: "362303354899e4d71a5accf22666225e"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.769954 7001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:24.770781 7016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:24.770990 7001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:24.771104 7001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "7cb7a70b24b44613bc9a85d87a878b13"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "711cd1b441ae7247635b77768407068e"
server_key_iv: "362303354899e4d71a5accf22666225e"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.771184 7001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:24.793537 7001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:24.796059 7001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:24.796239 7001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:24.801523 7001 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:40707
I20260928 17:56:24.801558 7078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:40707 every 8 connection(s)
I20260928 17:56:24.801944 7001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:24.802664 7079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:24.803869 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7001
I20260928 17:56:24.803974 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:24.804174 5305 external_mini_cluster.cc:1428] Setting key 5b36fb9e6b84586d49715d5cae2d2ca4
I20260928 17:56:24.805639 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Bootstrap starting.
I20260928 17:56:24.806316 7079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:24.806631 7079 log.cc:824] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:24.807459 7079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: No bootstrap required, opened a new log
I20260928 17:56:24.809813 7079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } }
I20260928 17:56:24.809979 7079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:24.810027 7079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cb7a70b24b44613bc9a85d87a878b13, State: Initialized, Role: FOLLOWER
I20260928 17:56:24.810173 7079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } }
I20260928 17:56:24.810281 7079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:24.810318 7079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:24.810361 7079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:24.811165 7079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } }
I20260928 17:56:24.811298 7079 leader_election.cc:302] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cb7a70b24b44613bc9a85d87a878b13; no voters:
I20260928 17:56:24.811430 7079 leader_election.cc:288] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:24.811499 7084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:24.811638 7084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [term 1 LEADER]: Becoming Leader. State: Replica: 7cb7a70b24b44613bc9a85d87a878b13, State: Running, Role: LEADER
I20260928 17:56:24.811669 7079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:24.811731 7084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } }
Sep 28 17:56:24 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:24.812075 7084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cb7a70b24b44613bc9a85d87a878b13. Latest consensus state: current_term: 1 leader_uuid: "7cb7a70b24b44613bc9a85d87a878b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } } }
I20260928 17:56:24.812158 7084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:24.812094 7085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cb7a70b24b44613bc9a85d87a878b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cb7a70b24b44613bc9a85d87a878b13" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40707 } } }
I20260928 17:56:24.812322 7085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13 [sys.catalog]: This master's current role is: LEADER
W20260928 17:56:24.813491 7100 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:56:24.813550 7100 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:56:24.813598 7090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:24.813755 7090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:24.815536 7090 catalog_manager.cc:1347] Generated new cluster ID: ef218fd60d6f48d781e6e3f9e7983e08
I20260928 17:56:24.815595 7090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:24.816932 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:24.804757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.5.46.126:40707)
0928 17:56:24.804946 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:24.804956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:24.804993 (+ 37us) server_negotiation.cc:407] Connection header received
0928 17:56:24.805532 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:24.805543 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:24.805606 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:24.805780 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:24.806571 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.807964 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.808750 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.809401 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.809528 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:24.812503 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:24.812507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:24.812507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:24.812519 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:24.814702 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:24.815139 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:24.815143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:24.815143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:24.815275 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:24.815517 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:24.815519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:24.815519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:24.815895 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:24.816036 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:24.816568 (+ 532us) server_negotiation.cc:299] Negotiation successful
0928 17:56:24.816719 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260928 17:56:24.820441 7090 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:24.821022 7090 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:24.840154 7090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Generated new TSK 0
I20260928 17:56:24.840410 7090 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:24.871708 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40707
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:24.956763 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:24.956970 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:24.956996 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:24.958117 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:24.958163 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:24.958184 7105 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 17:56:24.958195 7105 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:24.958212 7105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:24.959623 7105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:40707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:24.959821 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:24.960080 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:24.965269 7113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:24.965310 7110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:24.965514 7111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:24.965539 7105 server_base.cc:1034] running on GCE node
I20260928 17:56:24.965883 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:24.966179 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:24.967377 7105 hybrid_clock.cc:648] HybridClock initialized: now 1790618184967385 us; error 49 us; skew 500 ppm
Sep 28 17:56:24 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:24.969901 7105 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:24.970404 7105 webserver.cc:460] Webserver started at http://127.5.46.65:40153/ using document root <none> and password file <none>
I20260928 17:56:24.970651 7105 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:24.970722 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:24.970872 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:24.972062 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "da44f40313694826ae5777fba694ebb4"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "d7e820bd98fcd0a687c248e1c909534e"
server_key_iv: "a7a94827f38eb5b28d235ebade04f4eb"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.972432 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "da44f40313694826ae5777fba694ebb4"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "d7e820bd98fcd0a687c248e1c909534e"
server_key_iv: "a7a94827f38eb5b28d235ebade04f4eb"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.973824 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 17:56:24.974756 7120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:24.974967 7105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:24.975060 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "da44f40313694826ae5777fba694ebb4"
format_stamp: "Formatted at 2026-09-28 17:56:24 on dist-test-slave-z2t3"
server_key: "d7e820bd98fcd0a687c248e1c909534e"
server_key_iv: "a7a94827f38eb5b28d235ebade04f4eb"
server_key_version: "encryptionkey@0"
I20260928 17:56:24.975143 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:24.982115 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:24.984397 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:24.984571 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:24.985112 7105 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:24.985180 7105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:24.985227 7105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:24.985253 7105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:24.992101 7105 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33075
I20260928 17:56:24.992228 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33075 every 8 connection(s)
I20260928 17:56:24.992532 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:24 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:24.996207 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:24.985651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:43359 (local address 127.5.46.126:40707)
0928 17:56:24.985725 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:24.985728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:24.985932 (+ 204us) server_negotiation.cc:407] Connection header received
0928 17:56:24.986434 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:24.986436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:24.986449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:24.986557 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:24.987508 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.988423 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.990036 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.990385 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.990482 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:24.993288 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:24.993290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:24.993290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:24.993300 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:24.994844 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:24.995295 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:24.995296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:24.995296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:24.995349 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:24.995577 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:24.995578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:24.995578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:24.995685 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:24.995738 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:24.996066 (+ 328us) server_negotiation.cc:299] Negotiation successful
0928 17:56:24.996140 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 17:56:24.996460 7129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:24.985628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.65:43359)
0928 17:56:24.985798 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:24.985839 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:24.986329 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:24.986624 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:24.986628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:24.987086 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:24.987390 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:24.987396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.988556 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.988557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:24.989952 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:24.989955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:24.990493 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:24.990494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:24.990602 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:24.991439 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:24.991450 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:24.993164 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:24.994963 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:24.994971 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:24.994973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:24.995218 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:24.995410 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:24.995410 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:24.995411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:24.995519 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:24.995740 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:24.995743 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:24.995956 (+ 213us) client_negotiation.cc:769] Sending connection context
0928 17:56:24.996051 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 17:56:24.996186 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260928 17:56:24.997094 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7105
I20260928 17:56:24.997224 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:24.997174 7236 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40707
I20260928 17:56:24.997346 7236 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:24.997423 5305 external_mini_cluster.cc:1428] Setting key fdc20a97b2d6fa8cade862cbe3237964
I20260928 17:56:24.997548 7236 heartbeater.cc:511] Master 127.5.46.126:40707 requested a full tablet report, sending...
I20260928 17:56:24.998095 7033 ts_manager.cc:194] Registered new tserver with Master: da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075)
I20260928 17:56:24.999056 7033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:43359
I20260928 17:56:25.005504 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:24.998669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:51139 (local address 127.5.46.126:40707)
0928 17:56:24.998721 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:24.998724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:24.998902 (+ 178us) server_negotiation.cc:407] Connection header received
0928 17:56:24.998949 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:24.998949 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:24.998960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:24.999017 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:24.999407 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.000126 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.000897 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.001111 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.001211 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.002313 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.002315 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.002315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.002323 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.004194 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.004651 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.004652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.004653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.004714 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.004925 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.004926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.004926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.005059 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.005134 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.005308 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.005403 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 17:56:25.005508 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:24.998608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.65:51139)
0928 17:56:24.998814 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:24.998826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:24.998893 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:24.999021 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:24.999022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:24.999255 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:24.999328 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:24.999330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.000281 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.000282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.000784 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.000786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.001209 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.001210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.001297 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.001857 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.001862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.002197 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.004355 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.004356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.004357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.004554 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.004790 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.004791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.004791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.004857 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.005138 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.005139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.005191 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.005277 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.005402 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":143,"threads_started":1}
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:25.027402 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40707
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:25.112236 7245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:25.112447 7245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:25.112473 7245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:25.113610 7245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:25.113651 7245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:25.113672 7245 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 17:56:25.113684 7245 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:25.113701 7245 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:25.115088 7245 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:40707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:25.115274 7245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:25.115463 7245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:25.120589 7250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:25.120669 7245 server_base.cc:1034] running on GCE node
W20260928 17:56:25.120573 7253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:25.120569 7251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:25.121009 7245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:25.121249 7245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:25.122361 7245 hybrid_clock.cc:648] HybridClock initialized: now 1790618185122341 us; error 41 us; skew 500 ppm
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:25.125010 7245 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:25.125501 7245 webserver.cc:460] Webserver started at http://127.5.46.66:46269/ using document root <none> and password file <none>
I20260928 17:56:25.125749 7245 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:25.125823 7245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:25.125969 7245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:25.127053 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "5b162af626634f52bbe1c1562ab87861"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "dc2bb1e0cbd4b3f163c56aeae35715cd"
server_key_iv: "38a20a2ab3a95597a065b62f9ea4ad4a"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.127432 7245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "5b162af626634f52bbe1c1562ab87861"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "dc2bb1e0cbd4b3f163c56aeae35715cd"
server_key_iv: "38a20a2ab3a95597a065b62f9ea4ad4a"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.128878 7245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:25.129769 7260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.129952 7245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:25.130059 7245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "5b162af626634f52bbe1c1562ab87861"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "dc2bb1e0cbd4b3f163c56aeae35715cd"
server_key_iv: "38a20a2ab3a95597a065b62f9ea4ad4a"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.130136 7245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:25.144147 7245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:25.146598 7245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:25.146775 7245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:25.147322 7245 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:25.147390 7245 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.147436 7245 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:25.147480 7245 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.154661 7245 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:43755
I20260928 17:56:25.154742 7375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:43755 every 8 connection(s)
I20260928 17:56:25.155052 7245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:25.158248 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.147924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:57137 (local address 127.5.46.126:40707)
0928 17:56:25.148015 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.148018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.148179 (+ 161us) server_negotiation.cc:407] Connection header received
0928 17:56:25.148696 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.148698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.148712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.148802 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.149756 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.150627 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.152373 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.152657 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.152717 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.155755 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.155757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.155757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.155765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.157081 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.157457 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.157458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.157458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.157514 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.157729 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.157730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.157730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.157839 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.157895 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.158115 (+ 220us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.158179 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 17:56:25.158457 7269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.147828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.66:57137)
0928 17:56:25.148040 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.148082 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.148593 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.148921 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.148926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.149334 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:25.149660 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.149667 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.150743 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.150744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.152269 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.152272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.152729 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.152730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.152932 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.153635 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.153647 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.155607 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.157151 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.157157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.157158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.157373 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.157581 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.157582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.157582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.157674 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.157895 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.157899 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.158010 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.158092 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.158212 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260928 17:56:25.158965 7376 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40707
I20260928 17:56:25.159050 7376 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:25.159245 7376 heartbeater.cc:511] Master 127.5.46.126:40707 requested a full tablet report, sending...
I20260928 17:56:25.159602 7033 ts_manager.cc:194] Registered new tserver with Master: 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755)
I20260928 17:56:25.160208 7033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:57137
I20260928 17:56:25.162884 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7245
I20260928 17:56:25.162987 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:25.163159 5305 external_mini_cluster.cc:1428] Setting key f6019bcae1fe99db49ef40c0c97d3fe7
I20260928 17:56:25.166257 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.160224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:55169 (local address 127.5.46.126:40707)
0928 17:56:25.160282 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.160284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.160389 (+ 105us) server_negotiation.cc:407] Connection header received
0928 17:56:25.160447 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.160448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.160458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.160514 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.160854 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.161348 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.162106 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.162288 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.162343 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.163460 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.163462 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.163462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.163469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.165126 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.165556 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.165557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.165558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.165609 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.165809 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.165809 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.165810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.165933 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.165983 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.166141 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.166197 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:25.166428 7382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.160154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.66:55169)
0928 17:56:25.160325 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.160337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.160397 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.160521 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.160521 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.160718 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:25.160778 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.160780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.161456 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.161457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.162029 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.162031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.162365 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.162366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.162427 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.162912 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.162917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.163376 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.165213 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.165214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.165214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.165479 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.165673 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.165674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.165674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.165747 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.165990 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.165991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.166042 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.166124 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.166290 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:25.192134 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40707
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:25.275573 7385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:25.275795 7385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:25.275821 7385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:25.277143 7385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:25.277189 7385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:25.277210 7385 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 17:56:25.277222 7385 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:25.277275 7385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:25.278788 7385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:40707
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:25.279035 7385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:25.279277 7385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:25.284528 7393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:25.284528 7391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:25.284523 7390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:25.284780 7385 server_base.cc:1034] running on GCE node
I20260928 17:56:25.284973 7385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:25.285215 7385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:25.286377 7385 hybrid_clock.cc:648] HybridClock initialized: now 1790618185286367 us; error 34 us; skew 500 ppm
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:25.289227 7385 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:25.289712 7385 webserver.cc:460] Webserver started at http://127.5.46.67:37257/ using document root <none> and password file <none>
I20260928 17:56:25.289963 7385 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:25.290012 7385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:25.290144 7385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:25.291229 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "0f886011bd2949c0b3329241734a7866"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "aba093a97930b913d50774fddd460cbc"
server_key_iv: "a1a6044a77746867b5c372f312e8ade3"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.291594 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "0f886011bd2949c0b3329241734a7866"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "aba093a97930b913d50774fddd460cbc"
server_key_iv: "a1a6044a77746867b5c372f312e8ade3"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.293030 7385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:25.293828 7400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.294034 7385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:25.294142 7385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "0f886011bd2949c0b3329241734a7866"
format_stamp: "Formatted at 2026-09-28 17:56:25 on dist-test-slave-z2t3"
server_key: "aba093a97930b913d50774fddd460cbc"
server_key_iv: "a1a6044a77746867b5c372f312e8ade3"
server_key_version: "encryptionkey@0"
I20260928 17:56:25.294219 7385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:25.303187 7385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:25.305848 7385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:25.306025 7385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:25.306581 7385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:25.306643 7385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.306681 7385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:25.306730 7385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.314154 7385 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:43357
I20260928 17:56:25.314230 7515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:43357 every 8 connection(s)
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:25.314607 7385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:25.317178 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7385
I20260928 17:56:25.317306 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:25.317476 5305 external_mini_cluster.cc:1428] Setting key 818ab983531a9339ff2d5ed7f76c2696
I20260928 17:56:25.317996 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.307366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:35401 (local address 127.5.46.126:40707)
0928 17:56:25.307456 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.307459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.307670 (+ 211us) server_negotiation.cc:407] Connection header received
0928 17:56:25.308296 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.308298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.308313 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.308387 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.309347 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.310194 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.311756 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.312133 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.312209 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.314900 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.314901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.314902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.314910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.316476 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.316940 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.316941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.316942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.317007 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.317275 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.317277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.317277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.317421 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.317509 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.317810 (+ 301us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.317906 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260928 17:56:25.318200 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.307389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.67:35401)
0928 17:56:25.307552 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.307590 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.308194 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.308479 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.308483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.308873 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:25.309214 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.309220 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.310322 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.310323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.311680 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.311683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.312233 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.312234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.312341 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.313006 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.313016 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.314779 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.316594 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.316601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.316604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.316840 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.317085 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.317085 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.317086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.317199 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.317499 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.317505 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.317668 (+ 163us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.317776 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.317949 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260928 17:56:25.318653 7516 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40707
I20260928 17:56:25.318755 7516 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:25.318936 7516 heartbeater.cc:511] Master 127.5.46.126:40707 requested a full tablet report, sending...
I20260928 17:56:25.319367 7033 ts_manager.cc:194] Registered new tserver with Master: 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357)
I20260928 17:56:25.319954 7033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:35401
I20260928 17:56:25.325876 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.319815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.5.46.67:42919)
0928 17:56:25.320016 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.320030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.320107 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.320240 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.320241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.320412 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:25.320467 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.320469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.321127 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.321128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.321649 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.321650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.322054 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.322055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.322132 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.322785 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.322791 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.323100 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.324766 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.324767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.324768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.324948 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.325182 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.325183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.325184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.325242 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.325509 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.325510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.325560 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.325649 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.325768 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":100,"threads_started":1}
I20260928 17:56:25.325877 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.319927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:42919 (local address 127.5.46.126:40707)
0928 17:56:25.319968 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.319970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.320095 (+ 125us) server_negotiation.cc:407] Connection header received
0928 17:56:25.320156 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.320157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.320167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.320229 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.320548 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.321032 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.321739 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.321964 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.322052 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.323265 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.323266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.323267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.323274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.324665 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.325036 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.325037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.325038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.325099 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.325318 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.325319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.325319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.325440 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.325519 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.325675 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.325781 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:56:25.329581 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:25.354105 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.346070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.5.46.126:40707)
0928 17:56:25.346135 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.346138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.346208 (+ 70us) server_negotiation.cc:407] Connection header received
0928 17:56:25.346231 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.346232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.346241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.346310 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.346902 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.347404 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.348234 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.348429 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.348506 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.351006 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.351008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.351008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.351016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.352720 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.353092 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.353093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.353093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.353158 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.353473 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.353474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.353474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.353597 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.353711 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.353958 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.354034 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 17:56:25.356138 7033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:25.357721 7033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:25.364602 7533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:43755 (local address 127.0.0.1:52530)
0928 17:56:25.360378 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.360418 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.360497 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.360802 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.360806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.360809 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.360881 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.360882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362253 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.362254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.363326 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.363328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.364242 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.364243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.364343 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.364354 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.364431 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.364507 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":61,"threads_started":1}
I20260928 17:56:25.364621 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.5.46.66:43755)
0928 17:56:25.360574 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.360582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.360593 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:25.360636 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.360638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.360655 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.360796 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.360959 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362153 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.363394 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.364103 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.364180 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.364238 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.364455 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.364499 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260928 17:56:25.364790 7534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43357 (local address 127.0.0.1:52838)
0928 17:56:25.360797 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.360808 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.360852 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.361052 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.361053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.361055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.361094 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.361096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362423 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.362424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.363602 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.363604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.364518 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.364519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.364591 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.364595 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.364647 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.364700 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":88,"threads_started":1}
I20260928 17:56:25.365088 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52838 (local address 127.5.46.67:43357)
0928 17:56:25.360897 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.360901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.360911 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:25.360951 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.360953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.360967 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.361043 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.361189 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362336 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.363692 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.364422 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.364497 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.364553 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.364957 (+ 404us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.365003 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":73,"threads_started":1}
I20260928 17:56:25.365424 7310 tablet_service.cc:1467] Processing CreateTablet for tablet fdbeab983f224358a660793c63ac2411 (DEFAULT_TABLE table=test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.365808 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbeab983f224358a660793c63ac2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.365787 7450 tablet_service.cc:1467] Processing CreateTablet for tablet fdbeab983f224358a660793c63ac2411 (DEFAULT_TABLE table=test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.366142 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbeab983f224358a660793c63ac2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.366364 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33075 (local address 127.0.0.1:44858)
0928 17:56:25.360631 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.360641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.360703 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.361437 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.361437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.361439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.361490 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.361491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362560 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.362561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.363616 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.363617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.366106 (+ 2489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.366107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.366176 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.366194 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.366250 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.366296 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260928 17:56:25.366621 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.360617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.5.46.65:33075)
0928 17:56:25.361163 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.361167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.361176 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:25.361279 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.361281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.361294 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.361368 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.361562 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.362480 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.364604 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.365944 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.366018 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.366071 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.366484 (+ 413us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.366528 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":89,"threads_started":1}
I20260928 17:56:25.367329 7170 tablet_service.cc:1467] Processing CreateTablet for tablet fdbeab983f224358a660793c63ac2411 (DEFAULT_TABLE table=test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.367669 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbeab983f224358a660793c63ac2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.369014 7539 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Bootstrap starting.
I20260928 17:56:25.369638 7540 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Bootstrap starting.
I20260928 17:56:25.370306 7539 tablet_bootstrap.cc:654] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.370306 7540 tablet_bootstrap.cc:654] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.370664 7539 log.cc:824] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:25.370689 7540 log.cc:824] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:25.371541 7540 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: No bootstrap required, opened a new log
I20260928 17:56:25.371614 7540 ts_tablet_manager.cc:1397] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:25.371712 7539 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: No bootstrap required, opened a new log
I20260928 17:56:25.371770 7539 ts_tablet_manager.cc:1397] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:56:25.371776 7543 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Bootstrap starting.
I20260928 17:56:25.372337 7543 tablet_bootstrap.cc:654] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.372714 7543 log.cc:824] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:25.373377 7543 tablet_bootstrap.cc:492] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: No bootstrap required, opened a new log
I20260928 17:56:25.373467 7543 ts_tablet_manager.cc:1397] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:25.373741 7540 raft_consensus.cc:348] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.373734 7539 raft_consensus.cc:348] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.373908 7540 raft_consensus.cc:374] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.373908 7539 raft_consensus.cc:374] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.373945 7540 raft_consensus.cc:729] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.373945 7539 raft_consensus.cc:729] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.374084 7539 consensus_queue.cc:260] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.374084 7540 consensus_queue.cc:260] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.374380 7539 ts_tablet_manager.cc:1428] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 17:56:25.374382 7540 ts_tablet_manager.cc:1428] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:25.374511 7516 heartbeater.cc:503] Master 127.5.46.126:40707 was elected leader, sending a full tablet report...
I20260928 17:56:25.374548 7376 heartbeater.cc:503] Master 127.5.46.126:40707 was elected leader, sending a full tablet report...
I20260928 17:56:25.375492 7543 raft_consensus.cc:348] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.375655 7543 raft_consensus.cc:374] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.375706 7543 raft_consensus.cc:729] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da44f40313694826ae5777fba694ebb4, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.375826 7543 consensus_queue.cc:260] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.376142 7236 heartbeater.cc:503] Master 127.5.46.126:40707 was elected leader, sending a full tablet report...
I20260928 17:56:25.376158 7543 ts_tablet_manager.cc:1428] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260928 17:56:25.405656 7377 tablet.cc:2369] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:25.421636 7546 raft_consensus.cc:482] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:25.421794 7546 raft_consensus.cc:504] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.422179 7546 leader_election.cc:288] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755), da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075)
I20260928 17:56:25.425557 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.422591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:44677 (local address 127.5.46.65:33075)
0928 17:56:25.422640 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.422642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.422653 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:25.422684 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.422685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.422699 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.422759 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.422902 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.423501 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.424602 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.425209 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.425276 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.425295 (+ 19us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.425471 (+ 176us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.425505 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:25.425576 7409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.422412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33075 (local address 127.5.46.67:44677)
0928 17:56:25.422553 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.422567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.422618 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.422764 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.422765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.422766 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.422832 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.422834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.423595 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.423596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.424516 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.424518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.425281 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.425282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.425396 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.425400 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.425459 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.425499 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260928 17:56:25.426119 7548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.422412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:43755 (local address 127.5.46.67:60819)
0928 17:56:25.422598 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.422609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.422718 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.422897 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.422898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.422899 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.422979 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.422980 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.424076 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.424077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.425102 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.425104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.425881 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.425882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.425957 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.425960 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.426016 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.426052 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":27,"thread_start_us":89,"threads_started":1}
I20260928 17:56:25.426162 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.422501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:60819 (local address 127.5.46.66:43755)
0928 17:56:25.422570 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.422572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.422711 (+ 139us) server_negotiation.cc:407] Connection header received
0928 17:56:25.422773 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.422774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.422811 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.422864 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.423042 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.423502 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.425194 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.425767 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.425864 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.425876 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.426039 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.426084 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:56:25.426127 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbeab983f224358a660793c63ac2411" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4" is_pre_election: true
I20260928 17:56:25.426311 7190 raft_consensus.cc:2454] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:25.426447 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbeab983f224358a660793c63ac2411" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861" is_pre_election: true
I20260928 17:56:25.426558 7404 leader_election.cc:302] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:25.426587 7330 raft_consensus.cc:2454] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:25.426694 7546 raft_consensus.cc:2781] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:25.426789 7546 raft_consensus.cc:482] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:25.426827 7546 raft_consensus.cc:3037] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.427533 7546 raft_consensus.cc:504] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.427690 7546 leader_election.cc:288] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Requested vote from peers 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755), da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075)
I20260928 17:56:25.427832 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbeab983f224358a660793c63ac2411" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4"
I20260928 17:56:25.427874 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbeab983f224358a660793c63ac2411" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861"
I20260928 17:56:25.427953 7330 raft_consensus.cc:3037] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.427942 7190 raft_consensus.cc:3037] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.428647 7190 raft_consensus.cc:2454] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:25.428666 7330 raft_consensus.cc:2454] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:25.428805 7404 leader_election.cc:302] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:25.428905 7546 raft_consensus.cc:2781] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:25.429037 7546 raft_consensus.cc:686] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 1 LEADER]: Becoming Leader. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Running, Role: LEADER
I20260928 17:56:25.429131 7546 consensus_queue.cc:237] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.429978 7033 catalog_manager.cc:5495] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f886011bd2949c0b3329241734a7866 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } health_report { overall_health: HEALTHY } } }
Sep 28 17:56:25 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618184, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:25.468012 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.460850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.5.46.126:40707)
0928 17:56:25.460933 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.460935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.461028 (+ 93us) server_negotiation.cc:407] Connection header received
0928 17:56:25.461132 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.461133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.461145 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.461202 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:25.461707 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.462229 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.462927 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.463118 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.463202 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.465500 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:25.465502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:25.465502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:25.465509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:25.466739 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.467107 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.467108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.467109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.467164 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:25.467357 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:25.467358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:25.467358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:25.467472 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:25.467548 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.467709 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.467857 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 17:56:25.468086 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.460739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40707 (local address 127.0.0.1:45786)
0928 17:56:25.460943 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.460955 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.461049 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.461213 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.461213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.461568 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:25.461614 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.461616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.462321 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.462322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.462846 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.462848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.463203 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.463204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.463269 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.463809 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:25.463814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:25.465405 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:25.466834 (+ 1429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.466841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.466844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.467034 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.467237 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:25.467238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:25.467238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:25.467295 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:25.467538 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:25.467545 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:25.467600 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.467685 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.467946 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":150,"thread_start_us":95,"threads_started":1}
I20260928 17:56:25.470080 7033 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:45786:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 17:56:25.470263 7033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:25.471952 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c4d5a21fd411089a588758d1a8fff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e36f1fb2aef04da9ab1550e94fd67976]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:25.472117 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c4d5a21fd411089a588758d1a8fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.472141 7450 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c4d5a21fd411089a588758d1a8fff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e36f1fb2aef04da9ab1550e94fd67976]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:25.472329 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c4d5a21fd411089a588758d1a8fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.473588 7540 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: Bootstrap starting.
I20260928 17:56:25.473647 7539 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: Bootstrap starting.
I20260928 17:56:25.473743 7170 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1c4d5a21fd411089a588758d1a8fff (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e36f1fb2aef04da9ab1550e94fd67976]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:25.473925 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1c4d5a21fd411089a588758d1a8fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.474054 7539 tablet_bootstrap.cc:654] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.474349 7540 tablet_bootstrap.cc:654] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.474748 7539 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: No bootstrap required, opened a new log
I20260928 17:56:25.474818 7539 ts_tablet_manager.cc:1397] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:25.474988 7539 raft_consensus.cc:348] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.475068 7539 raft_consensus.cc:374] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.475093 7539 raft_consensus.cc:729] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.475145 7540 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: No bootstrap required, opened a new log
I20260928 17:56:25.475142 7539 consensus_queue.cc:260] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.475188 7540 ts_tablet_manager.cc:1397] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:25.475279 7539 ts_tablet_manager.cc:1428] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.475344 7540 raft_consensus.cc:348] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.475425 7540 raft_consensus.cc:374] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.475452 7540 raft_consensus.cc:729] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.475500 7540 consensus_queue.cc:260] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.475632 7540 ts_tablet_manager.cc:1428] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:25.475730 7545 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.475761 7546 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.475843 7545 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.475885 7546 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.475976 7543 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: Bootstrap starting.
I20260928 17:56:25.476804 7543 tablet_bootstrap.cc:654] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.477648 7543 tablet_bootstrap.cc:492] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: No bootstrap required, opened a new log
I20260928 17:56:25.477705 7543 ts_tablet_manager.cc:1397] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:25.477901 7543 raft_consensus.cc:348] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.477937 7546 raft_consensus.cc:482] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:25.477981 7543 raft_consensus.cc:374] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.477993 7546 raft_consensus.cc:504] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.478018 7543 raft_consensus.cc:729] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da44f40313694826ae5777fba694ebb4, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.478067 7543 consensus_queue.cc:260] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.478119 7546 leader_election.cc:288] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755)
I20260928 17:56:25.478155 7543 ts_tablet_manager.cc:1428] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:25.478293 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c4d5a21fd411089a588758d1a8fff" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4" is_pre_election: true
I20260928 17:56:25.478325 7547 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.478320 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c4d5a21fd411089a588758d1a8fff" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861" is_pre_election: true
I20260928 17:56:25.478403 7330 raft_consensus.cc:2454] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:25.478401 7547 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.478363 7190 raft_consensus.cc:2454] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:25.478574 7403 leader_election.cc:302] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, 5b162af626634f52bbe1c1562ab87861; no voters:
I20260928 17:56:25.478665 7546 raft_consensus.cc:2781] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:25.478698 7546 raft_consensus.cc:482] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:25.478745 7546 raft_consensus.cc:3037] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.479456 7546 raft_consensus.cc:504] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.479583 7546 leader_election.cc:288] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Requested vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755)
I20260928 17:56:25.479728 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c4d5a21fd411089a588758d1a8fff" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861"
I20260928 17:56:25.479805 7330 raft_consensus.cc:3037] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.479707 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1c4d5a21fd411089a588758d1a8fff" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4"
I20260928 17:56:25.479918 7190 raft_consensus.cc:3037] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.480553 7190 raft_consensus.cc:2454] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:25.480546 7330 raft_consensus.cc:2454] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:25.480711 7403 leader_election.cc:302] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, 5b162af626634f52bbe1c1562ab87861; no voters:
I20260928 17:56:25.480796 7546 raft_consensus.cc:2781] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:25.480841 7546 raft_consensus.cc:686] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [term 1 LEADER]: Becoming Leader. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Running, Role: LEADER
I20260928 17:56:25.480901 7546 consensus_queue.cc:237] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.481235 7549 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: TxnStatusTablet state changed. Reason: New leader 0f886011bd2949c0b3329241734a7866. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.481330 7549 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:25.481532 7561 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 17:56:25.481694 7561 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 17:56:25.481796 7032 catalog_manager.cc:5495] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f886011bd2949c0b3329241734a7866 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:25.489529 7556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.486306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43357 (local address 127.0.0.1:52844)
0928 17:56:25.486384 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.486403 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.486446 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.486759 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.486760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.486761 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:25.486806 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.486808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.487729 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.487730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.488438 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.488441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.488942 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.488943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.489020 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.489037 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.489365 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.489376 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.489421 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.489466 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 17:56:25.489564 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.486536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.5.46.67:43357)
0928 17:56:25.486604 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.486607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.486617 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:25.486648 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.486649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.486669 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.486745 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.486856 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.487603 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.488527 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.488839 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.488918 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.489144 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.489268 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.489358 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.489438 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.489462 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 17:56:25.491545 7190 raft_consensus.cc:1261] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.491626 7330 raft_consensus.cc:1261] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.491987 7546 consensus_queue.cc:1035] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.492132 7549 consensus_queue.cc:1035] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.493430 7545 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: TxnStatusTablet state changed. Reason: New leader 0f886011bd2949c0b3329241734a7866. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.493432 7547 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: TxnStatusTablet state changed. Reason: New leader 0f886011bd2949c0b3329241734a7866. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.493563 7547 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.493566 7545 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: This TxnStatusTablet replica's current role is: FOLLOWER
W20260928 17:56:25.494005 7237 tablet.cc:2369] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:25.494681 7546 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.494792 7546 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:25.494887 7546 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.494937 7546 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 0f886011bd2949c0b3329241734a7866: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:25.495568 7547 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.495654 7547 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P da44f40313694826ae5777fba694ebb4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.495720 7545 tablet_replica.cc:439] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } } }
I20260928 17:56:25.495803 7545 tablet_replica.cc:442] T 0e1c4d5a21fd411089a588758d1a8fff P 5b162af626634f52bbe1c1562ab87861: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:25.496563 7566 mvcc.cc:205] Tried to move back new op lower bound from 7334372087770697728 to 7334372087730294784. Current Snapshot: MvccSnapshot[applied={T|T < 7334372087770697728 or (T in {7334372087770697728})}]
I20260928 17:56:25.496670 7565 mvcc.cc:205] Tried to move back new op lower bound from 7334372087770697728 to 7334372087730294784. Current Snapshot: MvccSnapshot[applied={T|T < 7334372087770697728}]
I20260928 17:56:25.500166 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.497332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.5.46.67:43357)
0928 17:56:25.497371 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.497373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.497422 (+ 49us) server_negotiation.cc:407] Connection header received
0928 17:56:25.497508 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.497509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.497519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.497571 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.497702 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.498333 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.499452 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.499651 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.499708 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.499914 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.499948 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.499997 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.500078 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.500093 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 17:56:25.505693 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.502965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:52341 (local address 127.5.46.67:43357)
0928 17:56:25.503003 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.503005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.503014 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:25.503086 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.503086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.503096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.503136 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.503516 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.504097 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.504969 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.505175 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.505250 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.505430 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.505462 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.505533 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.505590 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.505607 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:25.505693 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.502918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43357 (local address 127.5.46.67:52341)
0928 17:56:25.502974 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.502985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.503029 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.503146 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.503147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.503148 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:25.503442 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.503444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.504186 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.504187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.504865 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.504867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.505249 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.505250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.505312 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.505325 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.505533 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.505535 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.505587 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.505629 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260928 17:56:25.508236 7330 raft_consensus.cc:1261] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.508236 7190 raft_consensus.cc:1261] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.508466 7546 consensus_queue.cc:1035] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.508637 7575 consensus_queue.cc:1035] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.523396 7033 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:25.523525 7033 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:25.524508 7033 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Sending DeleteTablet for 3 replicas of tablet fdbeab983f224358a660793c63ac2411
I20260928 17:56:25.524879 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbeab983f224358a660793c63ac2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:25 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52530
I20260928 17:56:25.524915 7170 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbeab983f224358a660793c63ac2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:25 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44858
I20260928 17:56:25.524899 7450 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbeab983f224358a660793c63ac2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:25 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52838
I20260928 17:56:25.525177 7580 tablet_replica.cc:330] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: stopping tablet replica
I20260928 17:56:25.525311 7580 raft_consensus.cc:2229] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:25.525323 7581 tablet_replica.cc:330] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: stopping tablet replica
I20260928 17:56:25.525437 7580 raft_consensus.cc:2258] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:25.525374 7033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:25.525508 7033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:25.525805 7581 raft_consensus.cc:2229] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:25.525995 7581 raft_consensus.cc:2258] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:25.526121 7580 ts_tablet_manager.cc:1905] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:25.526389 7582 tablet_replica.cc:330] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: stopping tablet replica
I20260928 17:56:25.526561 7582 raft_consensus.cc:2229] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:25.526598 7581 ts_tablet_manager.cc:1905] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:25.526870 7582 raft_consensus.cc:2258] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:25.526917 7170 tablet_service.cc:1467] Processing CreateTablet for tablet 6e829e66eb44411795a014fd114a6f51 (DEFAULT_TABLE table=test-table [id=96fe323093d5409ab914210cc8d86393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.527043 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e829e66eb44411795a014fd114a6f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.527505 7582 ts_tablet_manager.cc:1905] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:25.528093 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 6e829e66eb44411795a014fd114a6f51 (DEFAULT_TABLE table=test-table [id=96fe323093d5409ab914210cc8d86393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.528112 7450 tablet_service.cc:1467] Processing CreateTablet for tablet 6e829e66eb44411795a014fd114a6f51 (DEFAULT_TABLE table=test-table [id=96fe323093d5409ab914210cc8d86393]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:25.528213 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e829e66eb44411795a014fd114a6f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.528213 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e829e66eb44411795a014fd114a6f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:25.528384 7581 ts_tablet_manager.cc:1918] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:25.528453 7581 log.cc:1196] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/fdbeab983f224358a660793c63ac2411
I20260928 17:56:25.528718 7581 ts_tablet_manager.cc:1939] T fdbeab983f224358a660793c63ac2411 P 5b162af626634f52bbe1c1562ab87861: Deleting consensus metadata
I20260928 17:56:25.528730 7543 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Bootstrap starting.
I20260928 17:56:25.529139 7543 tablet_bootstrap.cc:654] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.529146 7019 catalog_manager.cc:4841] TS 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755): tablet fdbeab983f224358a660793c63ac2411 (table test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]) successfully deleted
I20260928 17:56:25.529531 7539 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Bootstrap starting.
I20260928 17:56:25.529590 7580 ts_tablet_manager.cc:1918] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:25.529611 7540 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Bootstrap starting.
I20260928 17:56:25.529661 7580 log.cc:1196] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/fdbeab983f224358a660793c63ac2411
I20260928 17:56:25.529929 7539 tablet_bootstrap.cc:654] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.529953 7580 ts_tablet_manager.cc:1939] T fdbeab983f224358a660793c63ac2411 P da44f40313694826ae5777fba694ebb4: Deleting consensus metadata
I20260928 17:56:25.530049 7540 tablet_bootstrap.cc:654] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:25.530351 7020 catalog_manager.cc:4841] TS da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075): tablet fdbeab983f224358a660793c63ac2411 (table test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]) successfully deleted
I20260928 17:56:25.530690 7539 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: No bootstrap required, opened a new log
I20260928 17:56:25.530690 7540 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: No bootstrap required, opened a new log
I20260928 17:56:25.530745 7539 ts_tablet_manager.cc:1397] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:25.530745 7540 ts_tablet_manager.cc:1397] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:25.530902 7540 raft_consensus.cc:348] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.530894 7539 raft_consensus.cc:348] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.530973 7539 raft_consensus.cc:374] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.530973 7540 raft_consensus.cc:374] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.530997 7539 raft_consensus.cc:729] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.530997 7540 raft_consensus.cc:729] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.531040 7539 consensus_queue.cc:260] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.531039 7540 consensus_queue.cc:260] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.531124 7540 ts_tablet_manager.cc:1428] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:25.531124 7539 ts_tablet_manager.cc:1428] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:25.532202 7582 ts_tablet_manager.cc:1918] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:25.532248 7543 tablet_bootstrap.cc:492] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: No bootstrap required, opened a new log
I20260928 17:56:25.532297 7543 ts_tablet_manager.cc:1397] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 17:56:25.532312 7582 log.cc:1196] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/fdbeab983f224358a660793c63ac2411
I20260928 17:56:25.532433 7543 raft_consensus.cc:348] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.532501 7543 raft_consensus.cc:374] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:25.532517 7543 raft_consensus.cc:729] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da44f40313694826ae5777fba694ebb4, State: Initialized, Role: FOLLOWER
I20260928 17:56:25.532552 7543 consensus_queue.cc:260] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.532622 7543 ts_tablet_manager.cc:1428] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:25.532624 7582 ts_tablet_manager.cc:1939] T fdbeab983f224358a660793c63ac2411 P 0f886011bd2949c0b3329241734a7866: Deleting consensus metadata
I20260928 17:56:25.533058 7019 catalog_manager.cc:4841] TS 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357): tablet fdbeab983f224358a660793c63ac2411 (table test-table [id=a0c94c56dc4a483f95f2961b8cb16bfa]) successfully deleted
W20260928 17:56:25.565333 7517 tablet.cc:2369] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:25.678946 7545 raft_consensus.cc:482] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:25.679059 7545 raft_consensus.cc:504] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.679414 7545 leader_election.cc:288] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357)
I20260928 17:56:25.683051 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.679826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:46249 (local address 127.5.46.65:33075)
0928 17:56:25.679885 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.679886 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.679895 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:25.679923 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.679924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.679934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.679999 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.680155 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.680926 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682079 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.682631 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682716 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.682727 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.682922 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.682972 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:56:25.683171 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.679734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:44177 (local address 127.5.46.67:43357)
0928 17:56:25.679789 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.679791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.679915 (+ 124us) server_negotiation.cc:407] Connection header received
0928 17:56:25.680019 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.680020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.680030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.680082 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:25.680235 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.680926 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682249 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.682805 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682884 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.682895 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.683057 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.683094 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 17:56:25.683187 7587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.679628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43357 (local address 127.5.46.66:44177)
0928 17:56:25.679837 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.679864 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.679938 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.680088 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.680089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.680092 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.680158 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.680160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.681020 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.681021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.682138 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.682141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.682885 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.682958 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.682969 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.683045 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.683091 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260928 17:56:25.683310 7586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.679628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33075 (local address 127.5.46.66:46249)
0928 17:56:25.679775 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.679786 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.679862 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.680025 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.680026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.680029 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:25.680092 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.680094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.681045 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.681046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.681982 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.681985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.682736 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.682737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.682811 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.682816 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.683180 (+ 364us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.683231 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":51,"thread_start_us":63,"threads_started":1}
I20260928 17:56:25.683598 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e829e66eb44411795a014fd114a6f51" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4" is_pre_election: true
I20260928 17:56:25.683702 7190 raft_consensus.cc:2454] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 0.
I20260928 17:56:25.683665 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e829e66eb44411795a014fd114a6f51" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f886011bd2949c0b3329241734a7866" is_pre_election: true
I20260928 17:56:25.683815 7470 raft_consensus.cc:2454] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 0.
I20260928 17:56:25.684072 7264 leader_election.cc:302] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b162af626634f52bbe1c1562ab87861, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:25.684227 7545 raft_consensus.cc:2781] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:25.684273 7545 raft_consensus.cc:482] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:25.684302 7545 raft_consensus.cc:3037] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.685205 7545 raft_consensus.cc:504] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.685379 7545 leader_election.cc:288] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 election: Requested vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357)
I20260928 17:56:25.685530 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e829e66eb44411795a014fd114a6f51" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f886011bd2949c0b3329241734a7866"
I20260928 17:56:25.685580 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e829e66eb44411795a014fd114a6f51" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4"
I20260928 17:56:25.685621 7470 raft_consensus.cc:3037] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.685653 7190 raft_consensus.cc:3037] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:25.686407 7190 raft_consensus.cc:2454] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 1.
I20260928 17:56:25.686407 7470 raft_consensus.cc:2454] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 1.
I20260928 17:56:25.686594 7263 leader_election.cc:302] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, 5b162af626634f52bbe1c1562ab87861; no voters:
I20260928 17:56:25.686750 7545 raft_consensus.cc:2781] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:25.686890 7545 raft_consensus.cc:686] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 1 LEADER]: Becoming Leader. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Running, Role: LEADER
I20260928 17:56:25.686967 7545 consensus_queue.cc:237] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:25.687793 7032 catalog_manager.cc:5495] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b162af626634f52bbe1c1562ab87861 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "5b162af626634f52bbe1c1562ab87861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:25.721055 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.718415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.5.46.66:43755)
0928 17:56:25.718470 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.718472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.718480 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:25.718514 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.718514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.718534 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.718622 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.718768 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.719351 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.720238 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.720451 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.720529 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.720697 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.720817 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.720903 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.720944 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.720966 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 17:56:25.726436 7538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.723586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:34151 (local address 127.5.46.67:43357)
0928 17:56:25.723625 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.723627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.723752 (+ 125us) server_negotiation.cc:407] Connection header received
0928 17:56:25.723828 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.723829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.723839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.723915 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.724056 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.724503 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.725700 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.725903 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.725989 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.726178 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.726213 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.726278 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.726338 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.726356 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 17:56:25.726469 7592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.723529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43357 (local address 127.5.46.66:34151)
0928 17:56:25.723669 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.723680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.723767 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.723918 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.723918 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.723920 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:25.723981 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.723983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.724584 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.724586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.725604 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.725607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.725992 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.725993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.726065 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.726084 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.726278 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.726281 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.726334 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.726377 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260928 17:56:25.728914 7190 raft_consensus.cc:1261] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Refusing update from remote peer 5b162af626634f52bbe1c1562ab87861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.728931 7470 raft_consensus.cc:1261] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Refusing update from remote peer 5b162af626634f52bbe1c1562ab87861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:25.729209 7588 consensus_queue.cc:1035] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.729372 7545 consensus_queue.cc:1035] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:25.731137 7564 mvcc.cc:205] Tried to move back new op lower bound from 7334372088743788544 to 7334372088574697472. Current Snapshot: MvccSnapshot[applied={T|T < 7334372088743788544}]
I20260928 17:56:25.737223 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.734388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:43755 (local address 127.5.46.67:53139)
0928 17:56:25.734450 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:25.734462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:25.734502 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:25.734642 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:25.734643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:25.734645 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:25.734691 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.734693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.735631 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.735633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.736363 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:25.736367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.736825 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.736826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:25.736877 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.736882 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.737058 (+ 176us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.737059 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:25.737103 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 17:56:25.737148 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 17:56:25.737408 7535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:25.734445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:53139 (local address 127.5.46.66:43755)
0928 17:56:25.734486 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:25.734488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:25.734497 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:25.734561 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:25.734562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:25.734575 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:25.734627 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:25.734781 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.735502 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.736454 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:25.736712 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:25.736790 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:25.736941 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:25.736981 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:25.737042 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:25.737332 (+ 290us) server_negotiation.cc:299] Negotiation successful
0928 17:56:25.737352 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 17:56:25.737660 7565 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 17:56:27.258544 7032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:27.258652 7032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45772:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:27.259527 7032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Sending DeleteTablet for 3 replicas of tablet 6e829e66eb44411795a014fd114a6f51
I20260928 17:56:27.259814 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e829e66eb44411795a014fd114a6f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52530
I20260928 17:56:27.259827 7170 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e829e66eb44411795a014fd114a6f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44858
I20260928 17:56:27.259902 7450 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e829e66eb44411795a014fd114a6f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52838
I20260928 17:56:27.260049 7620 tablet_replica.cc:330] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: stopping tablet replica
I20260928 17:56:27.260106 7620 raft_consensus.cc:2229] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:27.260154 7620 raft_consensus.cc:2258] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.260403 7620 ts_tablet_manager.cc:1905] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:27.261555 7621 tablet_replica.cc:330] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: stopping tablet replica
I20260928 17:56:27.261642 7621 raft_consensus.cc:2229] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:27.261703 7621 raft_consensus.cc:2258] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.261969 7620 ts_tablet_manager.cc:1918] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 17:56:27.262023 7620 log.cc:1196] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/6e829e66eb44411795a014fd114a6f51
I20260928 17:56:27.262012 7621 ts_tablet_manager.cc:1905] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:27.262245 7620 ts_tablet_manager.cc:1939] T 6e829e66eb44411795a014fd114a6f51 P da44f40313694826ae5777fba694ebb4: Deleting consensus metadata
I20260928 17:56:27.262466 7619 tablet_replica.cc:330] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: stopping tablet replica
I20260928 17:56:27.262562 7619 raft_consensus.cc:2229] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:27.262543 7020 catalog_manager.cc:4841] TS da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075): tablet 6e829e66eb44411795a014fd114a6f51 (table test-table [id=96fe323093d5409ab914210cc8d86393]) successfully deleted
I20260928 17:56:27.262670 7619 raft_consensus.cc:2258] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.262887 7619 ts_tablet_manager.cc:1905] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:27.264186 7619 ts_tablet_manager.cc:1918] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 17:56:27.264251 7619 log.cc:1196] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/6e829e66eb44411795a014fd114a6f51
I20260928 17:56:27.264499 7619 ts_tablet_manager.cc:1939] T 6e829e66eb44411795a014fd114a6f51 P 5b162af626634f52bbe1c1562ab87861: Deleting consensus metadata
I20260928 17:56:27.264844 7019 catalog_manager.cc:4841] TS 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755): tablet 6e829e66eb44411795a014fd114a6f51 (table test-table [id=96fe323093d5409ab914210cc8d86393]) successfully deleted
I20260928 17:56:27.265093 7621 ts_tablet_manager.cc:1918] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 17:56:27.265143 7621 log.cc:1196] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/6e829e66eb44411795a014fd114a6f51
I20260928 17:56:27.265381 7621 ts_tablet_manager.cc:1939] T 6e829e66eb44411795a014fd114a6f51 P 0f886011bd2949c0b3329241734a7866: Deleting consensus metadata
I20260928 17:56:27.265802 7019 catalog_manager.cc:4841] TS 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357): tablet 6e829e66eb44411795a014fd114a6f51 (table test-table [id=96fe323093d5409ab914210cc8d86393]) successfully deleted
Sep 28 17:56:27 dist-test-slave-z2t3 krb5kdc[6988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618187, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 17:56:27 dist-test-slave-z2t3 krb5kdc[6988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618187, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:27.285665 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:27.277041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45802 (local address 127.5.46.126:40707)
0928 17:56:27.277205 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:27.277207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:27.277316 (+ 109us) server_negotiation.cc:407] Connection header received
0928 17:56:27.277396 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:27.277397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:27.277407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:27.277469 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:27.278118 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.278787 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.279529 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.279744 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.279813 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:27.282467 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:27.282469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:27.282469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:27.282476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:27.284103 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:27.284567 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:27.284569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:27.284569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:27.284636 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:27.284912 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:27.284913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:27.284913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:27.285041 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:27.285129 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:27.285397 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 17:56:27.285517 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":112,"threads_started":1}
I20260928 17:56:27.287478 7032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:27.287668 7032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:27.289345 7170 tablet_service.cc:1467] Processing CreateTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (DEFAULT_TABLE table=test-table [id=56b25b7fb565427cbacbc103e8c78e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.289345 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (DEFAULT_TABLE table=test-table [id=56b25b7fb565427cbacbc103e8c78e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.289486 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.289486 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.290886 7630 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Bootstrap starting.
I20260928 17:56:27.290963 7450 tablet_service.cc:1467] Processing CreateTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (DEFAULT_TABLE table=test-table [id=56b25b7fb565427cbacbc103e8c78e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.291116 7450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.291365 7630 tablet_bootstrap.cc:654] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.291656 7631 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Bootstrap starting.
I20260928 17:56:27.292040 7630 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: No bootstrap required, opened a new log
I20260928 17:56:27.292107 7631 tablet_bootstrap.cc:654] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.292104 7630 ts_tablet_manager.cc:1397] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:27.292241 7630 raft_consensus.cc:348] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.292443 7630 raft_consensus.cc:374] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.292519 7630 raft_consensus.cc:729] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da44f40313694826ae5777fba694ebb4, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.292593 7633 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Bootstrap starting.
I20260928 17:56:27.292585 7630 consensus_queue.cc:260] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.292781 7630 ts_tablet_manager.cc:1428] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:27.292937 7633 tablet_bootstrap.cc:654] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.293111 7631 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: No bootstrap required, opened a new log
I20260928 17:56:27.293177 7631 ts_tablet_manager.cc:1397] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 17:56:27.293303 7631 raft_consensus.cc:348] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.293403 7631 raft_consensus.cc:374] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.293426 7631 raft_consensus.cc:729] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.293473 7631 consensus_queue.cc:260] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.293584 7631 ts_tablet_manager.cc:1428] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:27.293648 7633 tablet_bootstrap.cc:492] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: No bootstrap required, opened a new log
I20260928 17:56:27.293694 7633 ts_tablet_manager.cc:1397] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:27.293838 7633 raft_consensus.cc:348] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.293927 7633 raft_consensus.cc:374] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.293972 7633 raft_consensus.cc:729] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.294027 7633 consensus_queue.cc:260] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.294134 7633 ts_tablet_manager.cc:1428] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:27.542979 7600 raft_consensus.cc:482] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:27.543087 7600 raft_consensus.cc:504] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.543354 7600 leader_election.cc:288] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755)
I20260928 17:56:27.543661 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c7c095c23b4ce1aa1ab2d8d0044fc9" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861" is_pre_election: true
I20260928 17:56:27.543663 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c7c095c23b4ce1aa1ab2d8d0044fc9" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4" is_pre_election: true
I20260928 17:56:27.543768 7330 raft_consensus.cc:2454] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:27.543768 7190 raft_consensus.cc:2454] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 0.
I20260928 17:56:27.543962 7403 leader_election.cc:302] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, 5b162af626634f52bbe1c1562ab87861; no voters:
I20260928 17:56:27.544083 7600 raft_consensus.cc:2781] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:27.544157 7600 raft_consensus.cc:482] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:27.544188 7600 raft_consensus.cc:3037] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.544899 7600 raft_consensus.cc:504] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.545055 7600 leader_election.cc:288] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Requested vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755)
I20260928 17:56:27.545297 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c7c095c23b4ce1aa1ab2d8d0044fc9" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4"
I20260928 17:56:27.545245 7330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11c7c095c23b4ce1aa1ab2d8d0044fc9" candidate_uuid: "0f886011bd2949c0b3329241734a7866" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b162af626634f52bbe1c1562ab87861"
I20260928 17:56:27.545403 7190 raft_consensus.cc:3037] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.545403 7330 raft_consensus.cc:3037] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.545980 7190 raft_consensus.cc:2454] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:27.546090 7330 raft_consensus.cc:2454] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f886011bd2949c0b3329241734a7866 in term 1.
I20260928 17:56:27.546175 7404 leader_election.cc:302] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f886011bd2949c0b3329241734a7866, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:27.546265 7600 raft_consensus.cc:2781] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:27.546339 7600 raft_consensus.cc:686] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 1 LEADER]: Becoming Leader. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Running, Role: LEADER
I20260928 17:56:27.546397 7600 consensus_queue.cc:237] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.547094 7032 catalog_manager.cc:5495] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f886011bd2949c0b3329241734a7866 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "0f886011bd2949c0b3329241734a7866" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:27.580150 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:27.577885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.5.46.67:43357)
0928 17:56:27.578038 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:27.578040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:27.578048 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:27.578076 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:27.578077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:27.578087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:27.578146 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:27.578286 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.578795 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.579533 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.579748 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.579812 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:27.579939 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:27.579975 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:27.580017 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:27.580069 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 17:56:27.580087 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260928 17:56:27.582438 7330 raft_consensus.cc:1261] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:27.582484 7190 raft_consensus.cc:1261] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Refusing update from remote peer 0f886011bd2949c0b3329241734a7866: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:27.582649 7600 consensus_queue.cc:1035] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:27.582774 7597 consensus_queue.cc:1035] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:27.592573 7032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:27.592707 7032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:27.593396 7032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Sending DeleteTablet for 3 replicas of tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9
I20260928 17:56:27.593743 7449 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52838
I20260928 17:56:27.593750 7170 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44858
I20260928 17:56:27.593895 7621 tablet_replica.cc:330] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: stopping tablet replica
I20260928 17:56:27.593918 7620 tablet_replica.cc:330] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: stopping tablet replica
I20260928 17:56:27.593927 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:27 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52530
I20260928 17:56:27.593971 7621 raft_consensus.cc:2229] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:27.593995 7620 raft_consensus.cc:2229] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:27.594040 7621 raft_consensus.cc:2258] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.594026 7619 tablet_replica.cc:330] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: stopping tablet replica
I20260928 17:56:27.594058 7620 raft_consensus.cc:2258] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.594085 7619 raft_consensus.cc:2229] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:27.594122 7619 raft_consensus.cc:2258] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:27.594390 7621 ts_tablet_manager.cc:1905] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:27.594334 7032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 17:56:27.594478 7619 ts_tablet_manager.cc:1905] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 17:56:27.594539 7032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:27.594951 7620 ts_tablet_manager.cc:1905] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:27.596264 7449 tablet_service.cc:1467] Processing CreateTablet for tablet d063aea6083640e29c77c0b09e780fa7 (DEFAULT_TABLE table=test-table [id=1d4c161640a24330b4d1d2c6c9cc2fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.596275 7170 tablet_service.cc:1467] Processing CreateTablet for tablet d063aea6083640e29c77c0b09e780fa7 (DEFAULT_TABLE table=test-table [id=1d4c161640a24330b4d1d2c6c9cc2fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.596386 7170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d063aea6083640e29c77c0b09e780fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.596647 7619 ts_tablet_manager.cc:1918] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:27.596760 7619 log.cc:1196] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/11c7c095c23b4ce1aa1ab2d8d0044fc9
I20260928 17:56:27.596755 7620 ts_tablet_manager.cc:1918] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:27.596805 7620 log.cc:1196] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/11c7c095c23b4ce1aa1ab2d8d0044fc9
I20260928 17:56:27.597023 7619 ts_tablet_manager.cc:1939] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 5b162af626634f52bbe1c1562ab87861: Deleting consensus metadata
I20260928 17:56:27.597070 7620 ts_tablet_manager.cc:1939] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P da44f40313694826ae5777fba694ebb4: Deleting consensus metadata
I20260928 17:56:27.597402 7020 catalog_manager.cc:4841] TS da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075): tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (table test-table [id=56b25b7fb565427cbacbc103e8c78e4c]) successfully deleted
I20260928 17:56:27.597496 7310 tablet_service.cc:1467] Processing CreateTablet for tablet d063aea6083640e29c77c0b09e780fa7 (DEFAULT_TABLE table=test-table [id=1d4c161640a24330b4d1d2c6c9cc2fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:27.597616 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d063aea6083640e29c77c0b09e780fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.597625 7630 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4: Bootstrap starting.
I20260928 17:56:27.598223 7630 tablet_bootstrap.cc:654] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.598786 7631 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Bootstrap starting.
I20260928 17:56:27.598824 7019 catalog_manager.cc:4841] TS 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755): tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (table test-table [id=56b25b7fb565427cbacbc103e8c78e4c]) successfully deleted
I20260928 17:56:27.598862 7630 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4: No bootstrap required, opened a new log
I20260928 17:56:27.598941 7630 ts_tablet_manager.cc:1397] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:27.599076 7630 raft_consensus.cc:348] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.599166 7630 raft_consensus.cc:374] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.599190 7630 raft_consensus.cc:729] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da44f40313694826ae5777fba694ebb4, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.599156 7621 ts_tablet_manager.cc:1918] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:27.599165 7449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d063aea6083640e29c77c0b09e780fa7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:27.599231 7621 log.cc:1196] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/11c7c095c23b4ce1aa1ab2d8d0044fc9
I20260928 17:56:27.599227 7631 tablet_bootstrap.cc:654] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.599236 7630 consensus_queue.cc:260] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.599326 7630 ts_tablet_manager.cc:1428] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:27.599490 7621 ts_tablet_manager.cc:1939] T 11c7c095c23b4ce1aa1ab2d8d0044fc9 P 0f886011bd2949c0b3329241734a7866: Deleting consensus metadata
I20260928 17:56:27.599870 7019 catalog_manager.cc:4841] TS 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357): tablet 11c7c095c23b4ce1aa1ab2d8d0044fc9 (table test-table [id=56b25b7fb565427cbacbc103e8c78e4c]) successfully deleted
I20260928 17:56:27.600116 7631 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: No bootstrap required, opened a new log
I20260928 17:56:27.600171 7631 ts_tablet_manager.cc:1397] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:27.600304 7631 raft_consensus.cc:348] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.600404 7631 raft_consensus.cc:374] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.600440 7631 raft_consensus.cc:729] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.600493 7631 consensus_queue.cc:260] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.600601 7631 ts_tablet_manager.cc:1428] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:27.600639 7633 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Bootstrap starting.
I20260928 17:56:27.600962 7633 tablet_bootstrap.cc:654] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:27.601788 7633 tablet_bootstrap.cc:492] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: No bootstrap required, opened a new log
I20260928 17:56:27.601836 7633 ts_tablet_manager.cc:1397] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:27.601971 7633 raft_consensus.cc:348] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.602041 7633 raft_consensus.cc:374] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:27.602085 7633 raft_consensus.cc:729] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f886011bd2949c0b3329241734a7866, State: Initialized, Role: FOLLOWER
I20260928 17:56:27.602144 7633 consensus_queue.cc:260] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.602252 7633 ts_tablet_manager.cc:1428] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:27.698997 7610 raft_consensus.cc:482] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:27.699100 7610 raft_consensus.cc:504] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.699436 7610 leader_election.cc:288] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357)
I20260928 17:56:27.699697 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d063aea6083640e29c77c0b09e780fa7" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4" is_pre_election: true
I20260928 17:56:27.699697 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d063aea6083640e29c77c0b09e780fa7" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f886011bd2949c0b3329241734a7866" is_pre_election: true
I20260928 17:56:27.699818 7190 raft_consensus.cc:2454] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 0.
I20260928 17:56:27.699818 7470 raft_consensus.cc:2454] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 0.
I20260928 17:56:27.700037 7264 leader_election.cc:302] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b162af626634f52bbe1c1562ab87861, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:27.700165 7610 raft_consensus.cc:2781] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:27.700202 7610 raft_consensus.cc:482] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:27.700228 7610 raft_consensus.cc:3037] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.700924 7610 raft_consensus.cc:504] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.701071 7610 leader_election.cc:288] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 election: Requested vote from peers da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075), 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357)
I20260928 17:56:27.701220 7190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d063aea6083640e29c77c0b09e780fa7" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da44f40313694826ae5777fba694ebb4"
I20260928 17:56:27.701223 7470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d063aea6083640e29c77c0b09e780fa7" candidate_uuid: "5b162af626634f52bbe1c1562ab87861" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f886011bd2949c0b3329241734a7866"
I20260928 17:56:27.701318 7190 raft_consensus.cc:3037] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.701333 7470 raft_consensus.cc:3037] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:27.701941 7190 raft_consensus.cc:2454] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 1.
I20260928 17:56:27.702015 7470 raft_consensus.cc:2454] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b162af626634f52bbe1c1562ab87861 in term 1.
I20260928 17:56:27.702153 7264 leader_election.cc:302] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b162af626634f52bbe1c1562ab87861, da44f40313694826ae5777fba694ebb4; no voters:
I20260928 17:56:27.702272 7610 raft_consensus.cc:2781] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:27.702354 7610 raft_consensus.cc:686] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 1 LEADER]: Becoming Leader. State: Replica: 5b162af626634f52bbe1c1562ab87861, State: Running, Role: LEADER
I20260928 17:56:27.702419 7610 consensus_queue.cc:237] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } }
I20260928 17:56:27.703043 7032 catalog_manager.cc:5495] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b162af626634f52bbe1c1562ab87861 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "5b162af626634f52bbe1c1562ab87861" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b162af626634f52bbe1c1562ab87861" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 43755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:27.720340 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:27.717722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.5.46.66:43755)
0928 17:56:27.717935 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:27.717938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:27.717962 (+ 24us) server_negotiation.cc:407] Connection header received
0928 17:56:27.717987 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:27.717988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:27.717999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:27.718059 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:27.718211 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.718729 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.719590 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:27.719802 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:27.719905 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:27.720066 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:27.720106 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:27.720156 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:27.720216 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 17:56:27.720238 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260928 17:56:27.722754 7470 raft_consensus.cc:1261] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Refusing update from remote peer 5b162af626634f52bbe1c1562ab87861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:27.722754 7190 raft_consensus.cc:1261] T d063aea6083640e29c77c0b09e780fa7 P da44f40313694826ae5777fba694ebb4 [term 1 FOLLOWER]: Refusing update from remote peer 5b162af626634f52bbe1c1562ab87861: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:27.722980 7613 consensus_queue.cc:1035] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da44f40313694826ae5777fba694ebb4" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:27.723083 7610 consensus_queue.cc:1035] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f886011bd2949c0b3329241734a7866" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 17:56:27.727481 7602 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 17:56:29.245684 7031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:29.245826 7031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45802:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:29.246770 7031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7cb7a70b24b44613bc9a85d87a878b13: Sending DeleteTablet for 3 replicas of tablet d063aea6083640e29c77c0b09e780fa7
I20260928 17:56:29.247083 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet d063aea6083640e29c77c0b09e780fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:29 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52530
I20260928 17:56:29.247203 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7105
I20260928 17:56:29.247289 7170 tablet_service.cc:1514] Processing DeleteTablet for tablet d063aea6083640e29c77c0b09e780fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:29 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44858
I20260928 17:56:29.247741 7449 tablet_service.cc:1514] Processing DeleteTablet for tablet d063aea6083640e29c77c0b09e780fa7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:29 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:52838
I20260928 17:56:29.248174 7674 tablet_replica.cc:330] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: stopping tablet replica
I20260928 17:56:29.248251 7674 raft_consensus.cc:2229] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:29.248293 7674 raft_consensus.cc:2258] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:29.248596 7674 ts_tablet_manager.cc:1905] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:29.250821 7674 ts_tablet_manager.cc:1918] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 17:56:29.250902 7674 log.cc:1196] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/d063aea6083640e29c77c0b09e780fa7
I20260928 17:56:29.251149 7674 ts_tablet_manager.cc:1939] T d063aea6083640e29c77c0b09e780fa7 P 0f886011bd2949c0b3329241734a7866: Deleting consensus metadata
I20260928 17:56:29.251345 7673 tablet_replica.cc:330] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: stopping tablet replica
I20260928 17:56:29.251464 7673 raft_consensus.cc:2229] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:29.251560 7673 raft_consensus.cc:2258] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:29.251585 7019 catalog_manager.cc:4841] TS 0f886011bd2949c0b3329241734a7866 (127.5.46.67:43357): tablet d063aea6083640e29c77c0b09e780fa7 (table test-table [id=1d4c161640a24330b4d1d2c6c9cc2fa5]) successfully deleted
I20260928 17:56:29.252140 7673 ts_tablet_manager.cc:1905] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:29.254042 7673 ts_tablet_manager.cc:1918] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 17:56:29.254110 7673 log.cc:1196] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/d063aea6083640e29c77c0b09e780fa7
I20260928 17:56:29.254406 7673 ts_tablet_manager.cc:1939] T d063aea6083640e29c77c0b09e780fa7 P 5b162af626634f52bbe1c1562ab87861: Deleting consensus metadata
I20260928 17:56:29.254822 7019 catalog_manager.cc:4841] TS 5b162af626634f52bbe1c1562ab87861 (127.5.46.66:43755): tablet d063aea6083640e29c77c0b09e780fa7 (table test-table [id=1d4c161640a24330b4d1d2c6c9cc2fa5]) successfully deleted
W20260928 17:56:29.255466 7020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:33075 (error 108)
I20260928 17:56:29.255820 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7245
I20260928 17:56:29.260128 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:29.255965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33075 (local address 127.0.0.1:44864)
0928 17:56:29.260007 (+ 4042us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:29.260080 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4004,"thread_start_us":81,"threads_started":1}
W20260928 17:56:29.260281 7020 catalog_manager.cc:4584] TS da44f40313694826ae5777fba694ebb4 (127.5.46.65:33075): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d063aea6083640e29c77c0b09e780fa7: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33075: connect: Connection refused (error 111)
I20260928 17:56:29.262898 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7385
I20260928 17:56:29.269089 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7001
2026-09-28T17:56:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5819 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:29 dist-test-slave-z2t3 krb5kdc[7679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:29 dist-test-slave-z2t3 krb5kdc[7679](info): set up 2 sockets
Sep 28 17:56:29 dist-test-slave-z2t3 krb5kdc[7679](info): commencing operation
krb5kdc: starting...
W20260928 17:56:30.315820 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:30.447439 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33447
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:33447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:30.533649 7692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:30.533898 7692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:30.533937 7692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:30.535132 7692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:30.535202 7692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:30.535238 7692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:30.535286 7692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:30.535313 7692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:30.536873 7692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:33447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:30.537164 7692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:30.537415 7692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:30.542783 7697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.542884 7692 server_base.cc:1034] running on GCE node
W20260928 17:56:30.542800 7700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:30.542804 7698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.543455 7692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:30.543884 7692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:30.545058 7692 hybrid_clock.cc:648] HybridClock initialized: now 1790618190545037 us; error 32 us; skew 500 ppm
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:30.552879 7692 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:30.553503 7692 webserver.cc:460] Webserver started at http://127.5.46.126:41089/ using document root <none> and password file <none>
I20260928 17:56:30.553789 7692 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:30.553879 7692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:30.554044 7692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:30.555213 7692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "2e8e6aada9334575a7f9551d3df74880"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "018bf59d49c1dce3953d16b194e3ffc5"
server_key_iv: "bf11ab67bce14d784e1066e793b8ad5b"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.555611 7692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "2e8e6aada9334575a7f9551d3df74880"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "018bf59d49c1dce3953d16b194e3ffc5"
server_key_iv: "bf11ab67bce14d784e1066e793b8ad5b"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.557169 7692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:30.558007 7707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.558207 7692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:30.558312 7692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "2e8e6aada9334575a7f9551d3df74880"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "018bf59d49c1dce3953d16b194e3ffc5"
server_key_iv: "bf11ab67bce14d784e1066e793b8ad5b"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.558405 7692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:30.572682 7692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:30.575220 7692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:30.575429 7692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:30.579787 7692 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:33447
I20260928 17:56:30.579984 7759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:33447 every 8 connection(s)
I20260928 17:56:30.580294 7692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:30.581296 7760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:30.582664 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7692
I20260928 17:56:30.582767 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:30.582979 5305 external_mini_cluster.cc:1428] Setting key 2ba1dfb763ebf6c9bf173c9bbec9d5ef
I20260928 17:56:30.584179 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880: Bootstrap starting.
I20260928 17:56:30.584904 7760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:30.585283 7760 log.cc:824] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:30.586088 7760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880: No bootstrap required, opened a new log
I20260928 17:56:30.588534 7760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } }
I20260928 17:56:30.588725 7760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:30.588773 7760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e8e6aada9334575a7f9551d3df74880, State: Initialized, Role: FOLLOWER
I20260928 17:56:30.588902 7760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } }
I20260928 17:56:30.588990 7760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:30.589027 7760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:30.589073 7760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:30.589943 7760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } }
I20260928 17:56:30.590058 7760 leader_election.cc:302] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e8e6aada9334575a7f9551d3df74880; no voters:
I20260928 17:56:30.590301 7760 leader_election.cc:288] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:30.590344 7765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:30.590520 7765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [term 1 LEADER]: Becoming Leader. State: Replica: 2e8e6aada9334575a7f9551d3df74880, State: Running, Role: LEADER
I20260928 17:56:30.590615 7760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:30.590619 7765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } }
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:30.591045 7765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e8e6aada9334575a7f9551d3df74880. Latest consensus state: current_term: 1 leader_uuid: "2e8e6aada9334575a7f9551d3df74880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } } }
I20260928 17:56:30.591128 7765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:30.591073 7767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e8e6aada9334575a7f9551d3df74880" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e8e6aada9334575a7f9551d3df74880" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33447 } } }
I20260928 17:56:30.591316 7767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:30.592192 7772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:30.592717 7772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:30.594544 7772 catalog_manager.cc:1347] Generated new cluster ID: be5644f9750c4bcb96ef64d57704dbfd
I20260928 17:56:30.594604 7772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:30.595639 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:30.583639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.5.46.126:33447)
0928 17:56:30.583876 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:30.583887 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:30.583927 (+ 40us) server_negotiation.cc:407] Connection header received
0928 17:56:30.584411 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:30.584419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:30.584473 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:30.584599 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:30.585462 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.586495 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.587566 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.588097 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.588184 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:30.591606 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:30.591611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:30.591611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:30.591625 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:30.593866 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.594318 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.594322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.594322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.594449 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.594711 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.594712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.594712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.594962 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:30.595120 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:30.595252 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 17:56:30.595404 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":127,"threads_started":1}
I20260928 17:56:30.619174 7772 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:30.619764 7772 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:30.622363 7772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e8e6aada9334575a7f9551d3df74880: Generated new TSK 0
I20260928 17:56:30.622582 7772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:30.655102 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:30.750825 7786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:30.751048 7786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:30.751075 7786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:30.752291 7786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:30.752336 7786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:30.752375 7786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:30.753890 7786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:30.754117 7786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:30.754323 7786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:30.759629 7791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.759720 7786 server_base.cc:1034] running on GCE node
W20260928 17:56:30.759647 7794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:30.759645 7792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.760071 7786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:30.760461 7786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:30.761628 7786 hybrid_clock.cc:648] HybridClock initialized: now 1790618190761605 us; error 41 us; skew 500 ppm
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:30.764410 7786 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:30.764905 7786 webserver.cc:460] Webserver started at http://127.5.46.65:44701/ using document root <none> and password file <none>
I20260928 17:56:30.765117 7786 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:30.765161 7786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:30.765259 7786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:30.766355 7786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "1391795c1d334caf9e1cfc5403684084"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "32e8e2404f295c1b867c69109884b247"
server_key_iv: "778bdf9297ac519d83269e31c9f47cee"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.766649 7786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "1391795c1d334caf9e1cfc5403684084"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "32e8e2404f295c1b867c69109884b247"
server_key_iv: "778bdf9297ac519d83269e31c9f47cee"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.767998 7786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:30.768824 7801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.768997 7786 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:30.769093 7786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "1391795c1d334caf9e1cfc5403684084"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "32e8e2404f295c1b867c69109884b247"
server_key_iv: "778bdf9297ac519d83269e31c9f47cee"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.769151 7786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:30.779486 7786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:30.781944 7786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:30.782063 7786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:30.782250 7786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:30.782505 7786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:30.782536 7786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.782557 7786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:30.782570 7786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.788357 7786 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:44223
I20260928 17:56:30.788455 7914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:44223 every 8 connection(s)
I20260928 17:56:30.788794 7786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:30.791100 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7786
I20260928 17:56:30.791249 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:30.791447 5305 external_mini_cluster.cc:1428] Setting key 18c2c86a65037631ac56433ab2ae986d
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
I20260928 17:56:30.800282 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:30.789648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:41033 (local address 127.5.46.126:33447)
0928 17:56:30.789717 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:30.789719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:30.789949 (+ 230us) server_negotiation.cc:407] Connection header received
0928 17:56:30.790509 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:30.790511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:30.790523 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:30.790564 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:30.791983 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.792593 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.793930 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.794139 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.794207 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:30.797026 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:30.797028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:30.797028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:30.797037 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:30.798644 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.799137 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.799139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.799139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.799207 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.799505 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.799506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.799506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.799658 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:30.799726 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:30.800133 (+ 407us) server_negotiation.cc:299] Negotiation successful
0928 17:56:30.800210 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:56:30.800472 7917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:30.789624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33447 (local address 127.5.46.65:41033)
0928 17:56:30.789809 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:30.789853 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:30.790363 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:30.790645 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:30.790649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:30.791558 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:30.791905 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:30.791913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.792691 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.792692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:30.793833 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:30.793836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.794247 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.794248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:30.794399 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:30.795149 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:30.795160 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:30.796908 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:30.798775 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:30.798786 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:30.798789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:30.799049 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:30.799278 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:30.799279 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:30.799279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:30.799430 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:30.799738 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:30.799746 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:30.799972 (+ 226us) client_negotiation.cc:769] Sending connection context
0928 17:56:30.800103 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 17:56:30.800247 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":114,"threads_started":1}
I20260928 17:56:30.800994 7915 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33447
I20260928 17:56:30.801095 7915 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:30.801301 7915 heartbeater.cc:511] Master 127.5.46.126:33447 requested a full tablet report, sending...
I20260928 17:56:30.801949 7724 ts_manager.cc:194] Registered new tserver with Master: 1391795c1d334caf9e1cfc5403684084 (127.5.46.65:44223)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
I20260928 17:56:30.802729 7724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:41033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:30.818225 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:30.900983 7920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:30.901245 7920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:30.901286 7920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:30.902526 7920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:30.902607 7920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:30.902652 7920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:30.904235 7920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:30.904541 7920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:30.904806 7920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:30.910219 7925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:30.910313 7928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.910292 7920 server_base.cc:1034] running on GCE node
W20260928 17:56:30.910251 7926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:30.910676 7920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:30.910940 7920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:30.912107 7920 hybrid_clock.cc:648] HybridClock initialized: now 1790618190912096 us; error 33 us; skew 500 ppm
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:30.914804 7920 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:30.915336 7920 webserver.cc:460] Webserver started at http://127.5.46.66:37827/ using document root <none> and password file <none>
I20260928 17:56:30.915577 7920 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:30.915653 7920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:30.915838 7920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:30.916935 7920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "17ed97028cfc470c871427e29a8d9374"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "6907aface5a16fb911d54d43a5fb31fc"
server_key_iv: "f307c187c6594ec427a1a92c360e484c"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.917292 7920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "17ed97028cfc470c871427e29a8d9374"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "6907aface5a16fb911d54d43a5fb31fc"
server_key_iv: "f307c187c6594ec427a1a92c360e484c"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.918838 7920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:30.919629 7935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.919826 7920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:30.920001 7920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "17ed97028cfc470c871427e29a8d9374"
format_stamp: "Formatted at 2026-09-28 17:56:30 on dist-test-slave-z2t3"
server_key: "6907aface5a16fb911d54d43a5fb31fc"
server_key_iv: "f307c187c6594ec427a1a92c360e484c"
server_key_version: "encryptionkey@0"
I20260928 17:56:30.920090 7920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:30.933205 7920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:30.936102 7920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:30.936259 7920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:30.936522 7920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:30.936940 7920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:30.937011 7920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.937054 7920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:30.937083 7920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:30.943011 7920 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:38939
I20260928 17:56:30.943056 8048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:38939 every 8 connection(s)
I20260928 17:56:30.943419 7920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:30 dist-test-slave-z2t3 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618190, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:30.952549 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:30.944238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48233 (local address 127.5.46.126:33447)
0928 17:56:30.944298 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:30.944300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:30.944507 (+ 207us) server_negotiation.cc:407] Connection header received
0928 17:56:30.944990 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:30.944991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:30.945001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:30.945055 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:30.946025 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.946562 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.947557 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.947739 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.947791 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:30.949996 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:30.949997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:30.949997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:30.950004 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:30.951220 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.951601 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.951602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.951602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.951654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:30.951939 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:30.951939 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:30.951940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:30.952048 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:30.952103 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:30.952410 (+ 307us) server_negotiation.cc:299] Negotiation successful
0928 17:56:30.952480 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:56:30.952858 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:30.944211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33447 (local address 127.5.46.66:48233)
0928 17:56:30.944371 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:30.944408 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:30.944901 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:30.945110 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:30.945113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:30.945715 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:30.945927 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:30.945932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.946646 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.946647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:30.947479 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:30.947481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:30.947813 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:30.947814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:30.948017 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:30.948499 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:30.948507 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:30.949906 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:30.951294 (+ 1388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:30.951300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:30.951302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:30.951534 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:30.951714 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:30.951715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:30.951715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:30.951814 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:30.952131 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:30.952137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:30.952279 (+ 142us) client_negotiation.cc:769] Sending connection context
0928 17:56:30.952389 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 17:56:30.952537 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260928 17:56:30.953493 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 7920
I20260928 17:56:30.953480 8049 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33447
I20260928 17:56:30.953575 8049 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:30.953600 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:30.953791 8049 heartbeater.cc:511] Master 127.5.46.126:33447 requested a full tablet report, sending...
I20260928 17:56:30.953837 5305 external_mini_cluster.cc:1428] Setting key 432d8586cf8b45933bff67698fd11bd6
I20260928 17:56:30.954213 7724 ts_manager.cc:194] Registered new tserver with Master: 17ed97028cfc470c871427e29a8d9374 (127.5.46.66:38939)
I20260928 17:56:30.954730 7724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:48233
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:30.979282 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:31.072302 8054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:31.072556 8054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:31.072585 8054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:31.074169 8054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:31.074235 8054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:31.074271 8054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:31.076181 8054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:31.076495 8054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:31.076766 8054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:31.083096 8060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:31.083096 8062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:31.083120 8059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:31.083317 8054 server_base.cc:1034] running on GCE node
I20260928 17:56:31.083535 8054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:31.083756 8054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:31.084921 8054 hybrid_clock.cc:648] HybridClock initialized: now 1790618191084901 us; error 34 us; skew 500 ppm
Sep 28 17:56:31 dist-test-slave-z2t3 krb5kdc[7679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618191, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:31.087673 8054 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:31.088169 8054 webserver.cc:460] Webserver started at http://127.5.46.67:43907/ using document root <none> and password file <none>
I20260928 17:56:31.088433 8054 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:31.088479 8054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:31.088620 8054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:31.089690 8054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "fb61424019fe4a7da3db4fc780e20825"
format_stamp: "Formatted at 2026-09-28 17:56:31 on dist-test-slave-z2t3"
server_key: "99edb7837592c33bc03b60529e055c3b"
server_key_iv: "1b4d1ecfb7635088efb2d89c9e25d24c"
server_key_version: "encryptionkey@0"
I20260928 17:56:31.090054 8054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "fb61424019fe4a7da3db4fc780e20825"
format_stamp: "Formatted at 2026-09-28 17:56:31 on dist-test-slave-z2t3"
server_key: "99edb7837592c33bc03b60529e055c3b"
server_key_iv: "1b4d1ecfb7635088efb2d89c9e25d24c"
server_key_version: "encryptionkey@0"
I20260928 17:56:31.091718 8054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:31.092727 8069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:31.092935 8054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:31.093041 8054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "fb61424019fe4a7da3db4fc780e20825"
format_stamp: "Formatted at 2026-09-28 17:56:31 on dist-test-slave-z2t3"
server_key: "99edb7837592c33bc03b60529e055c3b"
server_key_iv: "1b4d1ecfb7635088efb2d89c9e25d24c"
server_key_version: "encryptionkey@0"
I20260928 17:56:31.093130 8054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:31.105031 8054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:31.107614 8054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:31.107823 8054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:31.108161 8054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:31.108502 8054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:31.108556 8054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:31.108597 8054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:31.108623 8054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:31.114588 8054 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:34641
I20260928 17:56:31.114650 8182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:34641 every 8 connection(s)
I20260928 17:56:31.115015 8054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:56:31 dist-test-slave-z2t3 krb5kdc[7679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618191, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:31.125042 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8054
I20260928 17:56:31.125128 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:31.125413 5305 external_mini_cluster.cc:1428] Setting key b3c79da95fb8e911ea114a78b42f7611
I20260928 17:56:31.126745 7763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:31.115819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:44181 (local address 127.5.46.126:33447)
0928 17:56:31.115905 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:31.115907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:31.116119 (+ 212us) server_negotiation.cc:407] Connection header received
0928 17:56:31.116649 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:31.116650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:31.116661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:31.116718 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:31.117717 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:31.118330 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:31.119431 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:31.119622 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:31.119685 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:31.122931 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:31.122933 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:31.122934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:31.122945 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:31.124951 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:31.125478 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:31.125481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:31.125481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:31.125558 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:31.125940 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:31.125941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:31.125941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:31.126064 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:31.126147 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:31.126547 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 17:56:31.126673 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 17:56:31.127005 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:31.115810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33447 (local address 127.5.46.67:44181)
0928 17:56:31.115986 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:31.116023 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:31.116552 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:31.116775 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:31.116778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:31.117385 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:31.117594 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:31.117599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:31.118438 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:31.118439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:31.119310 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:31.119312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:31.119719 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:31.119720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:31.120016 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:31.120633 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:31.120642 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:31.122716 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:31.125066 (+ 2350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:31.125079 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:31.125083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:31.125385 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:31.125617 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:31.125617 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:31.125618 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:31.125807 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:31.126138 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:31.126146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:31.126354 (+ 208us) client_negotiation.cc:769] Sending connection context
0928 17:56:31.126496 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 17:56:31.126713 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":105,"threads_started":1}
I20260928 17:56:31.127501 8183 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33447
I20260928 17:56:31.127604 8183 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:31.127913 8183 heartbeater.cc:511] Master 127.5.46.126:33447 requested a full tablet report, sending...
I20260928 17:56:31.128346 7724 ts_manager.cc:194] Registered new tserver with Master: fb61424019fe4a7da3db4fc780e20825 (127.5.46.67:34641)
I20260928 17:56:31.128726 7724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:44181
I20260928 17:56:31.137710 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 17:56:31.149991 7763 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:31.148848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49452 (local address 127.5.46.126:33447)
0928 17:56:31.148943 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:31.148945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:31.149048 (+ 103us) server_negotiation.cc:407] Connection header received
0928 17:56:31.149150 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:31.149151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:31.149162 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:31.149221 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:31.149907 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49452: BlockingRecv error: recv got EOF from 127.0.0.1:49452 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 17:56:31.151261 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7786
I20260928 17:56:31.155485 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7920
I20260928 17:56:31.160230 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8054
I20260928 17:56:31.164973 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 7692
2026-09-28T17:56:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1894 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:31 dist-test-slave-z2t3 krb5kdc[8195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:31 dist-test-slave-z2t3 krb5kdc[8195](info): set up 2 sockets
Sep 28 17:56:31 dist-test-slave-z2t3 krb5kdc[8195](info): commencing operation
krb5kdc: starting...
W20260928 17:56:32.204061 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:32.331800 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42877
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:32.415288 8208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:32.415529 8208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:32.415555 8208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:32.416880 8208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:32.416930 8208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:32.416944 8208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:32.416957 8208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:32.416970 8208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:32.418562 8208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:32.418839 8208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:32.419090 8208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:32.424332 8214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:32.424427 8208 server_base.cc:1034] running on GCE node
W20260928 17:56:32.424321 8216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:32.424345 8213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:32.424763 8208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:32.425129 8208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:32.426293 8208 hybrid_clock.cc:648] HybridClock initialized: now 1790618192426256 us; error 56 us; skew 500 ppm
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618192, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:32.429114 8208 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:32.429586 8208 webserver.cc:460] Webserver started at http://127.5.46.126:37561/ using document root <none> and password file <none>
I20260928 17:56:32.429798 8208 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:32.429847 8208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:32.429946 8208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:32.431156 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "14f45da6076749209ca9f8b131384122"
format_stamp: "Formatted at 2026-09-28 17:56:32 on dist-test-slave-z2t3"
server_key: "d267947f58c13c4d2ff7cd4713649f3e"
server_key_iv: "c4c2272999a974aac9b236cfc5b8e586"
server_key_version: "encryptionkey@0"
I20260928 17:56:32.431512 8208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "14f45da6076749209ca9f8b131384122"
format_stamp: "Formatted at 2026-09-28 17:56:32 on dist-test-slave-z2t3"
server_key: "d267947f58c13c4d2ff7cd4713649f3e"
server_key_iv: "c4c2272999a974aac9b236cfc5b8e586"
server_key_version: "encryptionkey@0"
I20260928 17:56:32.433192 8208 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 17:56:32.434031 8223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:32.434232 8208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:32.434322 8208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "14f45da6076749209ca9f8b131384122"
format_stamp: "Formatted at 2026-09-28 17:56:32 on dist-test-slave-z2t3"
server_key: "d267947f58c13c4d2ff7cd4713649f3e"
server_key_iv: "c4c2272999a974aac9b236cfc5b8e586"
server_key_version: "encryptionkey@0"
I20260928 17:56:32.434413 8208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:32.445943 8208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:32.448414 8208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:32.448536 8208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:32.452389 8208 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:42877
I20260928 17:56:32.452441 8275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:42877 every 8 connection(s)
I20260928 17:56:32.452785 8208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:32.453774 8276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:32.456776 8276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122: Bootstrap starting.
I20260928 17:56:32.457027 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8208
I20260928 17:56:32.457121 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:32.457304 5305 external_mini_cluster.cc:1428] Setting key f84dbe5572eb166705dde76d394eb514
I20260928 17:56:32.457597 8276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:32.457983 8276 log.cc:824] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:32.458894 8276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122: No bootstrap required, opened a new log
I20260928 17:56:32.461710 8276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } }
I20260928 17:56:32.461915 8276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:32.461959 8276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f45da6076749209ca9f8b131384122, State: Initialized, Role: FOLLOWER
I20260928 17:56:32.462090 8276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } }
I20260928 17:56:32.462178 8276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:32.462272 8276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:32.462335 8276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:32.463109 8276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } }
I20260928 17:56:32.463232 8276 leader_election.cc:302] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14f45da6076749209ca9f8b131384122; no voters:
I20260928 17:56:32.463406 8276 leader_election.cc:288] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:32.463477 8281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:32.463650 8276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:32.463680 8281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [term 1 LEADER]: Becoming Leader. State: Replica: 14f45da6076749209ca9f8b131384122, State: Running, Role: LEADER
I20260928 17:56:32.463831 8281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } }
I20260928 17:56:32.464327 8282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14f45da6076749209ca9f8b131384122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } } }
I20260928 17:56:32.464411 8282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:32.464582 8283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14f45da6076749209ca9f8b131384122. Latest consensus state: current_term: 1 leader_uuid: "14f45da6076749209ca9f8b131384122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14f45da6076749209ca9f8b131384122" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42877 } } }
I20260928 17:56:32.464641 8283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:32.464685 8290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:32.465349 8290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:32.466988 8290 catalog_manager.cc:1347] Generated new cluster ID: c28b15c9154048d0a78978441376aacc
I20260928 17:56:32.467046 8290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8195](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:32.478535 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:32.458345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57054 (local address 127.5.46.126:42877)
0928 17:56:32.458565 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:32.458576 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:32.458647 (+ 71us) server_negotiation.cc:407] Connection header received
0928 17:56:32.459321 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:32.459333 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:32.459412 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:32.459579 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:32.460589 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:32.462113 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:32.463234 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:32.463926 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:32.464032 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:32.474696 (+ 10664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:32.474702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:32.474702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:32.474718 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:32.476757 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:32.477192 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:32.477196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:32.477196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:32.477307 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:32.477534 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:32.477535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:32.477535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:32.477838 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:32.478010 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:32.478113 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 17:56:32.478279 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260928 17:56:32.479563 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:56:32.494773 8290 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:32.495343 8290 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:32.498302 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:32.493852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57070 (local address 127.5.46.126:42877)
0928 17:56:32.493905 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:32.493907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:32.494040 (+ 133us) server_negotiation.cc:407] Connection header received
0928 17:56:32.494114 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:32.494115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:32.494127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:32.494206 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:32.494607 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:32.495411 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:32.496571 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:32.496907 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:32.496977 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:32.497758 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:32.497759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:32.497760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:32.497996 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:32.498015 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:32.498022 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:32.498078 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:32.498166 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 17:56:32.498213 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:32.504529 8290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14f45da6076749209ca9f8b131384122: Generated new TSK 0
I20260928 17:56:32.504770 8290 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:56:32.513002 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8208
2026-09-28T17:56:32Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1345 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8309](info): set up 2 sockets
Sep 28 17:56:32 dist-test-slave-z2t3 krb5kdc[8309](info): commencing operation
krb5kdc: starting...
W20260928 17:56:33.605484 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:33 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:33.731244 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37759
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:37759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:33.812836 8322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:33.813125 8322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:33.813169 8322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:33.814455 8322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:33.814543 8322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:33.814565 8322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:33.814587 8322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:33.814615 8322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:33.816351 8322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:37759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:33.816661 8322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:33.816926 8322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:33.822283 8328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:33.822299 8330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:33.822340 8327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:33.822358 8322 server_base.cc:1034] running on GCE node
I20260928 17:56:33.822887 8322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:33.823231 8322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:33.824435 8322 hybrid_clock.cc:648] HybridClock initialized: now 1790618193824407 us; error 44 us; skew 500 ppm
Sep 28 17:56:33 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618193, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:33.827313 8322 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:33.827836 8322 webserver.cc:460] Webserver started at http://127.5.46.126:38759/ using document root <none> and password file <none>
I20260928 17:56:33.828109 8322 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:33.828152 8322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:33.828258 8322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:33.829329 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "b241272ac7ba4c27bc1a676023e78824"
format_stamp: "Formatted at 2026-09-28 17:56:33 on dist-test-slave-z2t3"
server_key: "68fa6e0139b450b18654014875cc28b0"
server_key_iv: "536e29d009593626dba64f60a18786e9"
server_key_version: "encryptionkey@0"
I20260928 17:56:33.829638 8322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "b241272ac7ba4c27bc1a676023e78824"
format_stamp: "Formatted at 2026-09-28 17:56:33 on dist-test-slave-z2t3"
server_key: "68fa6e0139b450b18654014875cc28b0"
server_key_iv: "536e29d009593626dba64f60a18786e9"
server_key_version: "encryptionkey@0"
I20260928 17:56:33.830986 8322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:33.831924 8337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:33.832140 8322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:33.832254 8322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "b241272ac7ba4c27bc1a676023e78824"
format_stamp: "Formatted at 2026-09-28 17:56:33 on dist-test-slave-z2t3"
server_key: "68fa6e0139b450b18654014875cc28b0"
server_key_iv: "536e29d009593626dba64f60a18786e9"
server_key_version: "encryptionkey@0"
I20260928 17:56:33.832353 8322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:33.845685 8322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:33.847961 8322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:33.848133 8322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:33.852051 8322 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:37759
I20260928 17:56:33.852092 8389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:37759 every 8 connection(s)
I20260928 17:56:33.852460 8322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:33.853459 8390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:33.855831 8390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824: Bootstrap starting.
I20260928 17:56:33.856396 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8322
I20260928 17:56:33.856530 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:33.856544 8390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:33.856720 5305 external_mini_cluster.cc:1428] Setting key 42d0442b139e7a9bac7e2b625fe6029a
I20260928 17:56:33.856887 8390 log.cc:824] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:33.857683 8390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824: No bootstrap required, opened a new log
I20260928 17:56:33.859732 8390 raft_consensus.cc:348] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } }
I20260928 17:56:33.859952 8390 raft_consensus.cc:374] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:33.860002 8390 raft_consensus.cc:729] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b241272ac7ba4c27bc1a676023e78824, State: Initialized, Role: FOLLOWER
I20260928 17:56:33.860143 8390 consensus_queue.cc:260] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } }
I20260928 17:56:33.860237 8390 raft_consensus.cc:388] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:33.860289 8390 raft_consensus.cc:482] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:33.860339 8390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:33.860898 8390 raft_consensus.cc:504] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } }
I20260928 17:56:33.861016 8390 leader_election.cc:302] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b241272ac7ba4c27bc1a676023e78824; no voters:
I20260928 17:56:33.861225 8390 leader_election.cc:288] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:33.861244 8395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:33.861370 8395 raft_consensus.cc:686] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [term 1 LEADER]: Becoming Leader. State: Replica: b241272ac7ba4c27bc1a676023e78824, State: Running, Role: LEADER
I20260928 17:56:33.861454 8395 consensus_queue.cc:237] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } }
I20260928 17:56:33.861492 8390 sys_catalog.cc:564] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:33.861817 8395 sys_catalog.cc:455] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b241272ac7ba4c27bc1a676023e78824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } } }
I20260928 17:56:33.861896 8395 sys_catalog.cc:458] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:33.861855 8396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b241272ac7ba4c27bc1a676023e78824. Latest consensus state: current_term: 1 leader_uuid: "b241272ac7ba4c27bc1a676023e78824" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b241272ac7ba4c27bc1a676023e78824" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37759 } } }
I20260928 17:56:33.862918 8401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:33.862053 8396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:33.863471 8401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 17:56:33 dist-test-slave-z2t3 krb5kdc[8309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:33.865281 8401 catalog_manager.cc:1347] Generated new cluster ID: 2938942c97ac4e22b5893810ba5310bd
I20260928 17:56:33.865329 8401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:33.867709 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:33.857364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.5.46.126:37759)
0928 17:56:33.857551 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:33.857561 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:33.857595 (+ 34us) server_negotiation.cc:407] Connection header received
0928 17:56:33.858099 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:33.858107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:33.858159 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:33.858308 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:33.858946 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:33.859812 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:33.860783 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:33.861317 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:33.861432 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:33.864343 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:33.864347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:33.864348 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:33.864361 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:33.866183 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:33.866569 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:33.866572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:33.866572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:33.866666 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:33.866868 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:33.866869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:33.866869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:33.867110 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:33.867277 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:33.867389 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 17:56:33.867530 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260928 17:56:33.880064 8401 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:33.880843 8401 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:33.895176 8401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b241272ac7ba4c27bc1a676023e78824: Generated new TSK 0
I20260928 17:56:33.895430 8401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:33.926429 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:34.027551 8416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:34.027791 8416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:34.027818 8416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:34.029218 8416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:34.029273 8416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:34.029302 8416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:34.030859 8416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:34.031158 8416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:34.031421 8416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:34.036975 8422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:34.036989 8421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:34.037025 8424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:34.037232 8416 server_base.cc:1034] running on GCE node
I20260928 17:56:34.037500 8416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:34.037802 8416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:34.039008 8416 hybrid_clock.cc:648] HybridClock initialized: now 1790618194039000 us; error 46 us; skew 500 ppm
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:34.042011 8416 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:34.042541 8416 webserver.cc:460] Webserver started at http://127.5.46.65:41933/ using document root <none> and password file <none>
I20260928 17:56:34.042768 8416 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:34.042810 8416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:34.042905 8416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:34.044107 8416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "39babd6c0e404d5baffeb220f50e5632"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "2e175e374bbeac698c1fef5882099761"
server_key_iv: "b9736e2189421d51ddc92184d2822c0e"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.044461 8416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "39babd6c0e404d5baffeb220f50e5632"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "2e175e374bbeac698c1fef5882099761"
server_key_iv: "b9736e2189421d51ddc92184d2822c0e"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.045900 8416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:34.046891 8431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.047117 8416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:34.047214 8416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "39babd6c0e404d5baffeb220f50e5632"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "2e175e374bbeac698c1fef5882099761"
server_key_iv: "b9736e2189421d51ddc92184d2822c0e"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.047276 8416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:34.064510 8416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:34.067001 8416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:34.067126 8416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:34.067327 8416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:34.067668 8416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:34.067705 8416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.067728 8416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:34.067773 8416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.073794 8416 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:34205
I20260928 17:56:34.073911 8544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:34205 every 8 connection(s)
I20260928 17:56:34.074230 8416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:34.082839 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8416
I20260928 17:56:34.082921 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:34.083153 5305 external_mini_cluster.cc:1428] Setting key 043d741d61948643a635c572a823bd4b
I20260928 17:56:34.084445 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.075039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:59039 (local address 127.5.46.126:37759)
0928 17:56:34.075111 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:34.075113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:34.075331 (+ 218us) server_negotiation.cc:407] Connection header received
0928 17:56:34.075911 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:34.075912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:34.075926 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:34.075976 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:34.076954 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.077469 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.078556 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.078794 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.078883 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.081244 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:34.081245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:34.081245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:34.081253 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:34.082686 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.083176 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.083177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.083178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.083261 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.083649 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.083650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.083650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.083783 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:34.083900 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:34.084279 (+ 379us) server_negotiation.cc:299] Negotiation successful
0928 17:56:34.084375 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 17:56:34.084790 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.075009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37759 (local address 127.5.46.65:59039)
0928 17:56:34.075196 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:34.075235 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:34.075753 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:34.076054 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:34.076058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:34.076627 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:34.076828 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.076833 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.077619 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.077620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.078460 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.078463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.078901 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.078902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.079032 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.079629 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:34.079637 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:34.081138 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:34.082778 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.082784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.082787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.083097 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.083392 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.083393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.083393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.083572 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.083890 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:34.083897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:34.084092 (+ 195us) client_negotiation.cc:769] Sending connection context
0928 17:56:34.084232 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 17:56:34.084488 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260928 17:56:34.085409 8545 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37759
I20260928 17:56:34.085541 8545 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:34.085750 8545 heartbeater.cc:511] Master 127.5.46.126:37759 requested a full tablet report, sending...
I20260928 17:56:34.086375 8354 ts_manager.cc:194] Registered new tserver with Master: 39babd6c0e404d5baffeb220f50e5632 (127.5.46.65:34205)
I20260928 17:56:34.087321 8354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:59039
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:34.110831 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:34.195420 8550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:34.195694 8550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:34.195721 8550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:34.197075 8550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:34.197141 8550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:34.197171 8550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:34.198726 8550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:34.199016 8550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:34.199267 8550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:34.204741 8555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:34.204802 8556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:34.204741 8558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:34.205135 8550 server_base.cc:1034] running on GCE node
I20260928 17:56:34.205353 8550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:34.205601 8550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:34.206763 8550 hybrid_clock.cc:648] HybridClock initialized: now 1790618194206751 us; error 40 us; skew 500 ppm
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:34.209512 8550 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:34.210053 8550 webserver.cc:460] Webserver started at http://127.5.46.66:38737/ using document root <none> and password file <none>
I20260928 17:56:34.210310 8550 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:34.210382 8550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:34.210539 8550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:34.211663 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "6213b4eca0a84d939b151f762f8efc75"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "75f410eb8df04f2210d421d54ffb5c46"
server_key_iv: "bbf06a6ee7bc374dddcb9c7ac32c005c"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.212085 8550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "6213b4eca0a84d939b151f762f8efc75"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "75f410eb8df04f2210d421d54ffb5c46"
server_key_iv: "bbf06a6ee7bc374dddcb9c7ac32c005c"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.213475 8550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:34.214228 8565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.214437 8550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:34.214545 8550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "6213b4eca0a84d939b151f762f8efc75"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "75f410eb8df04f2210d421d54ffb5c46"
server_key_iv: "bbf06a6ee7bc374dddcb9c7ac32c005c"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.214620 8550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:34.230067 8550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:34.232738 8550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:34.232975 8550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:34.233242 8550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:34.233600 8550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:34.233655 8550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.233696 8550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:34.233741 8550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.240272 8550 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:34281
I20260928 17:56:34.240336 8678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:34281 every 8 connection(s)
I20260928 17:56:34.240697 8550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:34.246380 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8550
I20260928 17:56:34.246479 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:34.246743 5305 external_mini_cluster.cc:1428] Setting key 5fde3ac1a7da65083afe0bff65d1766c
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:34.251330 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.241528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:45543 (local address 127.5.46.126:37759)
0928 17:56:34.241591 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:34.241593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:34.241789 (+ 196us) server_negotiation.cc:407] Connection header received
0928 17:56:34.242309 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:34.242310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:34.242321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:34.242393 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:34.243393 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.243914 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.244973 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.245239 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.245308 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.247755 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:34.247757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:34.247757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:34.247765 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:34.249491 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.250155 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.250156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.250156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.250228 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.250582 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.250583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.250583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.250700 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:34.250755 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:34.251143 (+ 388us) server_negotiation.cc:299] Negotiation successful
0928 17:56:34.251216 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:56:34.251662 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.241474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37759 (local address 127.5.46.66:45543)
0928 17:56:34.241644 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:34.241687 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:34.242195 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:34.242461 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:34.242464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:34.243077 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:34.243289 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.243294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.243991 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.243996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.244858 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.244860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.245336 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.245336 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.245449 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.245933 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:34.245941 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:34.247647 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:34.249598 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.249607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.249611 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.250039 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.250326 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.250327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.250327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.250508 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.250780 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:34.250787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:34.250990 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 17:56:34.251125 (+ 135us) client_negotiation.cc:240] Negotiation successful
0928 17:56:34.251334 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260928 17:56:34.252410 8679 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37759
I20260928 17:56:34.252539 8679 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:34.252768 8679 heartbeater.cc:511] Master 127.5.46.126:37759 requested a full tablet report, sending...
I20260928 17:56:34.253290 8354 ts_manager.cc:194] Registered new tserver with Master: 6213b4eca0a84d939b151f762f8efc75 (127.5.46.66:34281)
I20260928 17:56:34.253680 8354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:45543
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:34.275547 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:34.358650 8684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:34.358865 8684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:34.358889 8684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:34.360183 8684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:34.360232 8684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:34.360260 8684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:34.361755 8684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:34.361996 8684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:34.362201 8684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:34.367452 8690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:34.367450 8689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:34.367568 8684 server_base.cc:1034] running on GCE node
W20260928 17:56:34.367450 8692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:34.368191 8684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:34.368394 8684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:34.369527 8684 hybrid_clock.cc:648] HybridClock initialized: now 1790618194369517 us; error 26 us; skew 500 ppm
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:34.372140 8684 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:34.372637 8684 webserver.cc:460] Webserver started at http://127.5.46.67:34913/ using document root <none> and password file <none>
I20260928 17:56:34.372891 8684 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:34.372959 8684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:34.373108 8684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:34.374166 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "ffdd2529108e44fd8031dbe5a3c2b2ad"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "3be524af82fc5fc18767c7ad784c9a85"
server_key_iv: "f45cd48b70d717b0835b37360edf754b"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.374529 8684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "ffdd2529108e44fd8031dbe5a3c2b2ad"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "3be524af82fc5fc18767c7ad784c9a85"
server_key_iv: "f45cd48b70d717b0835b37360edf754b"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.375833 8684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:34.376631 8699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.376809 8684 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:34.376904 8684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "ffdd2529108e44fd8031dbe5a3c2b2ad"
format_stamp: "Formatted at 2026-09-28 17:56:34 on dist-test-slave-z2t3"
server_key: "3be524af82fc5fc18767c7ad784c9a85"
server_key_iv: "f45cd48b70d717b0835b37360edf754b"
server_key_version: "encryptionkey@0"
I20260928 17:56:34.376988 8684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:34.397115 8684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:34.399562 8684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:34.399734 8684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:34.400027 8684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:34.400352 8684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:34.400403 8684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.400445 8684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:34.400472 8684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:34.406262 8684 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:39063
I20260928 17:56:34.406298 8812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:39063 every 8 connection(s)
I20260928 17:56:34.406636 8684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:34.410912 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8684
I20260928 17:56:34.411008 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:34.411178 5305 external_mini_cluster.cc:1428] Setting key 11cf0e85a8d675ebad4ded875266b0af
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:34.416739 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.407384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:45721 (local address 127.5.46.126:37759)
0928 17:56:34.407442 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:34.407444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:34.407740 (+ 296us) server_negotiation.cc:407] Connection header received
0928 17:56:34.408327 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:34.408328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:34.408340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:34.408406 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:34.409368 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.409939 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.411148 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.411488 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.411559 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.414033 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:34.414035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:34.414035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:34.414042 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:34.415323 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.415754 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.415755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.415755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.415813 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.416141 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.416142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.416142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.416264 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:34.416322 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:34.416592 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 17:56:34.416661 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 17:56:34.416917 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.407382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37759 (local address 127.5.46.67:45721)
0928 17:56:34.407530 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:34.407567 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:34.408192 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:34.408479 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:34.408483 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:34.409076 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:34.409271 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.409276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.410024 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.410025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.410981 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:34.410984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.411587 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.411588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:34.411773 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.412433 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:34.412442 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:34.413929 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:34.415421 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.415427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.415429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.415670 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.415940 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:34.415942 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:34.415942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:34.416069 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:34.416327 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:34.416332 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:34.416454 (+ 122us) client_negotiation.cc:769] Sending connection context
0928 17:56:34.416542 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 17:56:34.416674 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260928 17:56:34.417464 8813 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37759
I20260928 17:56:34.417567 8813 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:34.417762 8813 heartbeater.cc:511] Master 127.5.46.126:37759 requested a full tablet report, sending...
I20260928 17:56:34.418159 8354 ts_manager.cc:194] Registered new tserver with Master: ffdd2529108e44fd8031dbe5a3c2b2ad (127.5.46.67:39063)
I20260928 17:56:34.418617 8354 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:45721
I20260928 17:56:34.423135 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618194, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:34.446893 8394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:34.439328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.5.46.126:37759)
0928 17:56:34.439386 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:34.439388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:34.439440 (+ 52us) server_negotiation.cc:407] Connection header received
0928 17:56:34.439504 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:34.439505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:34.439517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:34.439578 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:34.440313 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.440803 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.441648 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:34.441855 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:34.441923 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:34.444445 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:34.444447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:34.444447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:34.444454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:34.445911 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.446231 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.446232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.446233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.446281 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:34.446448 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:34.446449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:34.446449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:34.446584 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:34.446659 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:34.446762 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 17:56:34.446823 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260928 17:56:34.447146 8354 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50036
I20260928 17:56:34.447546 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8416
I20260928 17:56:34.451742 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8550
I20260928 17:56:34.456357 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8684
I20260928 17:56:34.470182 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8322
2026-09-28T17:56:34Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1960 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8825](info): set up 2 sockets
Sep 28 17:56:34 dist-test-slave-z2t3 krb5kdc[8825](info): commencing operation
krb5kdc: starting...
W20260928 17:56:35.527871 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:35 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:35.651497 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34171
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:34171
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:35.732393 8838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:35.732654 8838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:35.732694 8838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:35.733893 8838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:35.733976 8838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:35.733999 8838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:35.734020 8838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:35.734050 8838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:35.735626 8838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:34171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:35.735958 8838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:35.736202 8838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:35.741417 8843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:35.741453 8844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:35.741492 8838 server_base.cc:1034] running on GCE node
W20260928 17:56:35.741453 8846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:35.741971 8838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:35.742305 8838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:35.743487 8838 hybrid_clock.cc:648] HybridClock initialized: now 1790618195743453 us; error 52 us; skew 500 ppm
Sep 28 17:56:35 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618195, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:35.753120 8838 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:35.753643 8838 webserver.cc:460] Webserver started at http://127.5.46.126:33109/ using document root <none> and password file <none>
I20260928 17:56:35.753892 8838 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:35.753969 8838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:35.754127 8838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:35.755316 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "46c3e9845260405b9df0c2cffb845c30"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "b4aabd8ee1533359a34df434318eadce"
server_key_iv: "e3c294370150a24daee455cf59cfe309"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.755728 8838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "46c3e9845260405b9df0c2cffb845c30"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "b4aabd8ee1533359a34df434318eadce"
server_key_iv: "e3c294370150a24daee455cf59cfe309"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.757294 8838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:35.758167 8853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:35.758452 8838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:35.758564 8838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "46c3e9845260405b9df0c2cffb845c30"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "b4aabd8ee1533359a34df434318eadce"
server_key_iv: "e3c294370150a24daee455cf59cfe309"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.758647 8838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:35.770781 8838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:35.773396 8838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:35.773578 8838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:35.777664 8838 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:34171
I20260928 17:56:35.777706 8905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:34171 every 8 connection(s)
I20260928 17:56:35.778249 8838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:35.779018 8906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:35.781522 8906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30: Bootstrap starting.
I20260928 17:56:35.782168 8906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:35.782486 8906 log.cc:824] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:35.783196 8906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30: No bootstrap required, opened a new log
I20260928 17:56:35.784854 8906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } }
I20260928 17:56:35.785007 8906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:35.785030 8906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46c3e9845260405b9df0c2cffb845c30, State: Initialized, Role: FOLLOWER
I20260928 17:56:35.785141 8906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } }
I20260928 17:56:35.785225 8906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:35.785262 8906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:35.785337 8906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:35.786298 8906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } }
I20260928 17:56:35.786432 8906 leader_election.cc:302] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46c3e9845260405b9df0c2cffb845c30; no voters:
I20260928 17:56:35.786623 8906 leader_election.cc:288] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:35.786725 8909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:35.786895 8909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [term 1 LEADER]: Becoming Leader. State: Replica: 46c3e9845260405b9df0c2cffb845c30, State: Running, Role: LEADER
I20260928 17:56:35.786988 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8838
I20260928 17:56:35.786990 8909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } }
I20260928 17:56:35.787091 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:35.787307 5305 external_mini_cluster.cc:1428] Setting key 9e8097a4cb7919738967de1e1ba487e4
I20260928 17:56:35.787384 8910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46c3e9845260405b9df0c2cffb845c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } } }
I20260928 17:56:35.787464 8906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:35.787508 8911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46c3e9845260405b9df0c2cffb845c30. Latest consensus state: current_term: 1 leader_uuid: "46c3e9845260405b9df0c2cffb845c30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46c3e9845260405b9df0c2cffb845c30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34171 } } }
I20260928 17:56:35.787554 8911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:35.787470 8910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:35.787834 8913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:35.788663 8913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:35.790581 8913 catalog_manager.cc:1347] Generated new cluster ID: 39b2bd4d361a400ea0f6627c0a28e5f7
I20260928 17:56:35.790638 8913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:35 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:35.799794 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:35.787988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.5.46.126:34171)
0928 17:56:35.788201 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:35.788213 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:35.788385 (+ 172us) server_negotiation.cc:407] Connection header received
0928 17:56:35.788521 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:35.788529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:35.788613 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:35.788748 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:35.789651 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.790955 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.792121 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.792767 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.792871 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:35.795777 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:35.795784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:35.795784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:35.795802 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:35.797980 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:35.798425 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:35.798429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:35.798429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:35.798567 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:35.798773 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:35.798773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:35.798774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:35.799134 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:35.799317 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:35.799409 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 17:56:35.799563 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":75,"threads_started":1}
I20260928 17:56:35.802125 8913 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:35.802659 8913 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:35.815073 8913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46c3e9845260405b9df0c2cffb845c30: Generated new TSK 0
I20260928 17:56:35.815300 8913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:35.847091 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:35.929946 8932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:35.930213 8932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:35.930254 8932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:35.931617 8932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:35.931674 8932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:35.931720 8932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:35.933328 8932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:35.933629 8932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:35.933887 8932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:35.939328 8937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:35.939343 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:35.939615 8932 server_base.cc:1034] running on GCE node
W20260928 17:56:35.939327 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:35.940023 8932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:35.940313 8932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:35.941502 8932 hybrid_clock.cc:648] HybridClock initialized: now 1790618195941480 us; error 35 us; skew 500 ppm
Sep 28 17:56:35 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618195, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:35.944303 8932 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:35.944813 8932 webserver.cc:460] Webserver started at http://127.5.46.65:46291/ using document root <none> and password file <none>
I20260928 17:56:35.945070 8932 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:35.945143 8932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:35.945305 8932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:35.946419 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "428e616103bc4556869c09dfac245562"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "26d3d393ba465437e1a272d67068340c"
server_key_iv: "1c649c5cdf08ce2028d2654f564b1beb"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.946810 8932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "428e616103bc4556869c09dfac245562"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "26d3d393ba465437e1a272d67068340c"
server_key_iv: "1c649c5cdf08ce2028d2654f564b1beb"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.948395 8932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:35.949254 8947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:35.949443 8932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:35.949556 8932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "428e616103bc4556869c09dfac245562"
format_stamp: "Formatted at 2026-09-28 17:56:35 on dist-test-slave-z2t3"
server_key: "26d3d393ba465437e1a272d67068340c"
server_key_iv: "1c649c5cdf08ce2028d2654f564b1beb"
server_key_version: "encryptionkey@0"
I20260928 17:56:35.949643 8932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:35.984364 8932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:35.986828 8932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:35.987013 8932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:35.987264 8932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:35.987589 8932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:35.987651 8932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:35.987695 8932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:35.987753 8932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:35.994022 8932 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33735
I20260928 17:56:35.994073 9060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33735 every 8 connection(s)
I20260928 17:56:35.994403 8932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:35 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618195, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:36.002861 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 8932
I20260928 17:56:36.002981 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:36.003168 5305 external_mini_cluster.cc:1428] Setting key 0cf9f9b9906c7e1dcb8858fc5a421e26
I20260928 17:56:36.005015 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:35.995165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:41045 (local address 127.5.46.126:34171)
0928 17:56:35.995215 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:35.995217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:35.995429 (+ 212us) server_negotiation.cc:407] Connection header received
0928 17:56:35.996387 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:35.996389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:35.996400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:35.996465 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:35.997110 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.997639 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.998717 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.998926 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.998984 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.001440 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:36.001441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:36.001442 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:36.001451 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:36.003246 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.003832 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.003834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.003834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.003925 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.004210 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.004211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.004212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.004373 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:36.004463 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:36.004771 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 17:56:36.004872 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:36.005050 9063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:35.995123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34171 (local address 127.5.46.65:41045)
0928 17:56:35.995280 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:35.995324 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:35.996265 (+ 941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:35.996520 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:35.996523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:35.996792 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:35.996985 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:35.996989 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.997754 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.997755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:35.998621 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:35.998623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:35.999012 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:35.999012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:35.999119 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:35.999634 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:35.999643 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:36.001336 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:36.003361 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.003371 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.003373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.003713 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.004005 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.004006 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.004006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.004142 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.004484 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:36.004490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:36.004622 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 17:56:36.004727 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 17:56:36.004872 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":97,"threads_started":1}
I20260928 17:56:36.005926 9061 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34171
I20260928 17:56:36.006019 9061 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:36.006283 9061 heartbeater.cc:511] Master 127.5.46.126:34171 requested a full tablet report, sending...
I20260928 17:56:36.006867 8870 ts_manager.cc:194] Registered new tserver with Master: 428e616103bc4556869c09dfac245562 (127.5.46.65:33735)
I20260928 17:56:36.007530 8870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:41045
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:36.031430 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:36.114137 9066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:36.114392 9066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:36.114420 9066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:36.115584 9066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:36.115629 9066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:36.115658 9066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:36.117332 9066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:36.117549 9066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:36.117743 9066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:36.122926 9074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:36.122939 9072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:36.122961 9071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:36.123189 9066 server_base.cc:1034] running on GCE node
I20260928 17:56:36.123580 9066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:36.123803 9066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:36.124987 9066 hybrid_clock.cc:648] HybridClock initialized: now 1790618196124966 us; error 41 us; skew 500 ppm
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:36.127596 9066 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:36.128161 9066 webserver.cc:460] Webserver started at http://127.5.46.66:43223/ using document root <none> and password file <none>
I20260928 17:56:36.128407 9066 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:36.128476 9066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:36.128633 9066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:36.129731 9066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "b569f62d361245bdb9886a72ec316022"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "4f343c6e3f97f26c6bf3d8c93b2e6542"
server_key_iv: "cad46abf0c5b231bfa6d91bde0513a2b"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.130100 9066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "b569f62d361245bdb9886a72ec316022"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "4f343c6e3f97f26c6bf3d8c93b2e6542"
server_key_iv: "cad46abf0c5b231bfa6d91bde0513a2b"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.131524 9066 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:36.132393 9081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.132619 9066 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:36.132736 9066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "b569f62d361245bdb9886a72ec316022"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "4f343c6e3f97f26c6bf3d8c93b2e6542"
server_key_iv: "cad46abf0c5b231bfa6d91bde0513a2b"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.132812 9066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:36.142536 9066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:36.144943 9066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:36.145117 9066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:36.145380 9066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:36.145701 9066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:36.145756 9066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.145798 9066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:36.145826 9066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.152076 9066 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:35963
I20260928 17:56:36.152118 9194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:35963 every 8 connection(s)
I20260928 17:56:36.152447 9066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:36.157179 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9066
I20260928 17:56:36.157305 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:36.157478 5305 external_mini_cluster.cc:1428] Setting key 651e164415bdd84641d9f2e311044f68
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:36.162619 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.153241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:47937 (local address 127.5.46.126:34171)
0928 17:56:36.153326 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:36.153329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:36.153477 (+ 148us) server_negotiation.cc:407] Connection header received
0928 17:56:36.154595 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:36.154597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:36.154608 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:36.154667 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:36.155352 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.155958 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.157042 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.157246 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.157353 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.159587 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:36.159589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:36.159589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:36.159597 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:36.161135 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.161585 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.161586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.161587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.161651 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.161941 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.161942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.161942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.162066 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:36.162147 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:36.162438 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 17:56:36.162527 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 17:56:36.162842 9197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.153157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34171 (local address 127.5.46.66:47937)
0928 17:56:36.153307 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:36.153365 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:36.154484 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:36.154729 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:36.154733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:36.155025 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:36.155248 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:36.155253 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.156056 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.156057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:36.156929 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:36.156931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.157365 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.157366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:36.157496 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.158085 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:36.158094 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:36.159488 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:36.161234 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.161240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.161243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.161486 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.161730 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.161730 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.161731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.161844 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.162149 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:36.162156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:36.162300 (+ 144us) client_negotiation.cc:769] Sending connection context
0928 17:56:36.162419 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 17:56:36.162585 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260928 17:56:36.163301 9195 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34171
I20260928 17:56:36.163420 9195 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:36.163628 9195 heartbeater.cc:511] Master 127.5.46.126:34171 requested a full tablet report, sending...
I20260928 17:56:36.164049 8870 ts_manager.cc:194] Registered new tserver with Master: b569f62d361245bdb9886a72ec316022 (127.5.46.66:35963)
I20260928 17:56:36.164443 8870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:47937
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:36.186612 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:36.270236 9200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:36.270448 9200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:36.270473 9200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:36.271625 9200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:36.271664 9200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:36.271690 9200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:36.273128 9200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:36.273319 9200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:36.273492 9200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:36.278589 9205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:36.278733 9206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:36.278846 9200 server_base.cc:1034] running on GCE node
W20260928 17:56:36.278589 9208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:36.279121 9200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:36.279367 9200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:36.280512 9200 hybrid_clock.cc:648] HybridClock initialized: now 1790618196280493 us; error 38 us; skew 500 ppm
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:36.283210 9200 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:36.283704 9200 webserver.cc:460] Webserver started at http://127.5.46.67:36669/ using document root <none> and password file <none>
I20260928 17:56:36.283979 9200 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:36.284026 9200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:36.284194 9200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:36.285290 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "c83c0198dfb340f4a0e35465198291ec"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "76b906711f31c5d9191f1b8295bdd294"
server_key_iv: "21ac7eff3f8ba0e7f20acaac19128088"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.285646 9200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "c83c0198dfb340f4a0e35465198291ec"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "76b906711f31c5d9191f1b8295bdd294"
server_key_iv: "21ac7eff3f8ba0e7f20acaac19128088"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.286959 9200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:36.287811 9215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.288028 9200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:36.288166 9200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "c83c0198dfb340f4a0e35465198291ec"
format_stamp: "Formatted at 2026-09-28 17:56:36 on dist-test-slave-z2t3"
server_key: "76b906711f31c5d9191f1b8295bdd294"
server_key_iv: "21ac7eff3f8ba0e7f20acaac19128088"
server_key_version: "encryptionkey@0"
I20260928 17:56:36.288257 9200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:36.301431 9200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:36.304116 9200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:36.304253 9200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:36.304461 9200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:36.304822 9200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:36.304865 9200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.304893 9200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:36.304915 9200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:36.311233 9200 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:41695
I20260928 17:56:36.311288 9328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:41695 every 8 connection(s)
I20260928 17:56:36.311676 9200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:36.312075 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9200
I20260928 17:56:36.312176 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:36.312413 5305 external_mini_cluster.cc:1428] Setting key 5c932c5b351beff3333531a8bf97f8be
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:36.322237 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.312669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:56019 (local address 127.5.46.126:34171)
0928 17:56:36.312762 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:36.312765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:36.312990 (+ 225us) server_negotiation.cc:407] Connection header received
0928 17:56:36.314039 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:36.314040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:36.314051 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:36.314110 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:36.314847 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.315480 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.316608 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.316812 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.316879 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.319271 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:36.319273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:36.319273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:36.319282 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:36.320777 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.321219 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.321220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.321221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.321284 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.321546 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.321547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.321547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.321670 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:36.321749 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:36.322045 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 17:56:36.322127 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 17:56:36.322451 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.312662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34171 (local address 127.5.46.67:56019)
0928 17:56:36.312847 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:36.312891 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:36.313915 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:36.314245 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:36.314249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:36.314520 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:36.314727 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:36.314732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.315587 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.315588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:36.316506 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:36.316508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.316937 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.316938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:36.317051 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.317696 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:36.317705 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:36.319163 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:36.320882 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.320889 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.320891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.321122 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.321352 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:36.321352 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:36.321353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:36.321472 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:36.321739 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:36.321745 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:36.321896 (+ 151us) client_negotiation.cc:769] Sending connection context
0928 17:56:36.322023 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 17:56:36.322194 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260928 17:56:36.322993 9329 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34171
I20260928 17:56:36.323112 9329 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:36.323310 9329 heartbeater.cc:511] Master 127.5.46.126:34171 requested a full tablet report, sending...
I20260928 17:56:36.323812 8870 ts_manager.cc:194] Registered new tserver with Master: c83c0198dfb340f4a0e35465198291ec (127.5.46.67:41695)
I20260928 17:56:36.324342 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:56:36.324347 8870 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:56019
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:36.348285 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.340001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.5.46.65:33735)
0928 17:56:36.340163 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:36.340168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:36.340187 (+ 19us) server_negotiation.cc:407] Connection header received
0928 17:56:36.340214 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:36.340216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:36.340229 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:36.340339 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:36.340882 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.341644 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.342481 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.342822 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.342883 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.345446 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:36.345454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:36.345455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:36.345462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:36.347124 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.347484 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.347486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.347487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.347548 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.347722 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.347722 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.347723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.347899 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:36.348022 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:36.348099 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 17:56:36.348163 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[8825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:36.357169 8917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:36.349720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.5.46.126:34171)
0928 17:56:36.349892 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:36.349894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:36.349929 (+ 35us) server_negotiation.cc:407] Connection header received
0928 17:56:36.350028 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:36.350029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:36.350040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:36.350117 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:36.350616 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.351100 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.351813 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:36.352116 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:36.352200 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:36.354577 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:36.354579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:36.354579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:36.354587 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:36.356064 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.356454 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.356454 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.356455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.356516 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:36.356731 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:36.356732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:36.356732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:36.356881 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:36.356956 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:36.357047 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 17:56:36.357105 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
W20260928 17:56:36.357441 8870 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41362
I20260928 17:56:36.357867 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8932
I20260928 17:56:36.363116 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9066
I20260928 17:56:36.367712 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9200
I20260928 17:56:36.371984 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 8838
2026-09-28T17:56:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1900 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[9341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[9341](info): set up 2 sockets
Sep 28 17:56:36 dist-test-slave-z2t3 krb5kdc[9341](info): commencing operation
krb5kdc: starting...
W20260928 17:56:37.412042 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:37 dist-test-slave-z2t3 krb5kdc[9341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:37.536157 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41199
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:41199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:37.618182 9354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:37.618450 9354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:37.618494 9354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:37.619688 9354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:37.619776 9354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:37.619798 9354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:37.619819 9354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:37.619867 9354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:37.621399 9354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:41199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:37.621693 9354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:37.621958 9354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:37.626972 9362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:37.626974 9359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:37.627020 9360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:37.627482 9354 server_base.cc:1034] running on GCE node
I20260928 17:56:37.627830 9354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:37.628242 9354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:37.629437 9354 hybrid_clock.cc:648] HybridClock initialized: now 1790618197629423 us; error 45 us; skew 500 ppm
Sep 28 17:56:37 dist-test-slave-z2t3 krb5kdc[9341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618197, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:37.632411 9354 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:37.632850 9354 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:37.632927 9354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:37.633093 9354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:37.634258 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "a3ebf290043a444c9c38b427c6d26ed1"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "9ed9ce8a20a7d4a68ebfd39d4be28c99"
server_key_iv: "6dc37f6090e4d4742efd5758e271112a"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.634675 9354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "a3ebf290043a444c9c38b427c6d26ed1"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "9ed9ce8a20a7d4a68ebfd39d4be28c99"
server_key_iv: "6dc37f6090e4d4742efd5758e271112a"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.636173 9354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 17:56:37.636983 9368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:37.637177 9354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:37.637283 9354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "a3ebf290043a444c9c38b427c6d26ed1"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "9ed9ce8a20a7d4a68ebfd39d4be28c99"
server_key_iv: "6dc37f6090e4d4742efd5758e271112a"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.637365 9354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:37.649096 9354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:37.651233 9354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:37.651415 9354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:37.655799 9354 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:41199
I20260928 17:56:37.655817 9420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:41199 every 8 connection(s)
I20260928 17:56:37.656159 9354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:37.657063 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:37.659502 9421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: Bootstrap starting.
I20260928 17:56:37.660142 9421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:37.660450 9421 log.cc:824] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:37.661204 9421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: No bootstrap required, opened a new log
I20260928 17:56:37.661273 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9354
I20260928 17:56:37.661382 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:37.661584 5305 external_mini_cluster.cc:1428] Setting key b4f3e4a00a8dfe8ca495f9b761c8a6b3
I20260928 17:56:37.663916 9421 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } }
I20260928 17:56:37.664098 9421 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:37.664144 9421 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3ebf290043a444c9c38b427c6d26ed1, State: Initialized, Role: FOLLOWER
I20260928 17:56:37.664328 9421 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } }
I20260928 17:56:37.664425 9421 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:37.664471 9421 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:37.664521 9421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:37.665128 9421 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } }
I20260928 17:56:37.665258 9421 leader_election.cc:302] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3ebf290043a444c9c38b427c6d26ed1; no voters:
I20260928 17:56:37.665480 9421 leader_election.cc:288] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:37.665510 9426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:37.665618 9426 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [term 1 LEADER]: Becoming Leader. State: Replica: a3ebf290043a444c9c38b427c6d26ed1, State: Running, Role: LEADER
I20260928 17:56:37.665701 9426 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } }
I20260928 17:56:37.665763 9421 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:37.666045 9426 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3ebf290043a444c9c38b427c6d26ed1. Latest consensus state: current_term: 1 leader_uuid: "a3ebf290043a444c9c38b427c6d26ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } } }
I20260928 17:56:37.666060 9427 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3ebf290043a444c9c38b427c6d26ed1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3ebf290043a444c9c38b427c6d26ed1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41199 } } }
I20260928 17:56:37.666124 9426 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:37.666174 9427 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:37.666544 9432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:37.667290 9432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:37.669318 9432 catalog_manager.cc:1347] Generated new cluster ID: cb884622713642f9b4fdde586f27ab17
I20260928 17:56:37.669382 9432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:37 dist-test-slave-z2t3 krb5kdc[9341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:37.673875 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:37.662275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.5.46.126:41199)
0928 17:56:37.662468 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:37.662479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:37.662520 (+ 41us) server_negotiation.cc:407] Connection header received
0928 17:56:37.663011 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:37.663019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:37.663073 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:37.663246 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:37.663977 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.665083 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.666467 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.667125 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.667242 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:37.670182 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:37.670186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:37.670187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:37.670200 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:37.672202 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:37.672635 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:37.672638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:37.672638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:37.672746 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:37.672975 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:37.672975 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:37.672976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:37.673269 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:37.673410 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:37.673536 (+ 126us) server_negotiation.cc:299] Negotiation successful
0928 17:56:37.673663 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260928 17:56:37.688692 9432 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:37.689347 9432 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:37.695205 9432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: Generated new TSK 0
I20260928 17:56:37.695474 9432 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:37.731204 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:37.832240 9447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:37.832459 9447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:37.832484 9447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:37.833582 9447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:37.833618 9447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:37.833644 9447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:37.835204 9447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:37.835414 9447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:37.835592 9447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:37.840973 9452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:37.840987 9453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:37.841009 9455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:37.841277 9447 server_base.cc:1034] running on GCE node
I20260928 17:56:37.841564 9447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:37.841823 9447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:37.842963 9447 hybrid_clock.cc:648] HybridClock initialized: now 1790618197842941 us; error 42 us; skew 500 ppm
Sep 28 17:56:37 dist-test-slave-z2t3 krb5kdc[9341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618197, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:37.845604 9447 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:37.845923 9447 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:37.845968 9447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:37.846058 9447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:37.847091 9447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "00128c96108244ad9c0e58a83e446df2"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "04bdc709623d43682551e3248101514f"
server_key_iv: "ab3caeaccf46e8c562002806ee71ffd0"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.847398 9447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "00128c96108244ad9c0e58a83e446df2"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "04bdc709623d43682551e3248101514f"
server_key_iv: "ab3caeaccf46e8c562002806ee71ffd0"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.848750 9447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:37.849530 9461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:37.849730 9447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:37.849823 9447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "00128c96108244ad9c0e58a83e446df2"
format_stamp: "Formatted at 2026-09-28 17:56:37 on dist-test-slave-z2t3"
server_key: "04bdc709623d43682551e3248101514f"
server_key_iv: "ab3caeaccf46e8c562002806ee71ffd0"
server_key_version: "encryptionkey@0"
I20260928 17:56:37.849882 9447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:37.862231 9447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:37.864604 9447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:37.864723 9447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:37.864910 9447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:37.865247 9447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:37.865278 9447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:37.865301 9447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:37.865314 9447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:37.870976 9447 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36439
I20260928 17:56:37.871055 9574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36439 every 8 connection(s)
I20260928 17:56:37.871366 9447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:37.877391 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9447
I20260928 17:56:37.877524 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:37.877722 5305 external_mini_cluster.cc:1428] Setting key 2e97ed23481769420f7bc90eab2b7b65
Sep 28 17:56:37 dist-test-slave-z2t3 krb5kdc[9341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618197, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:37.882336 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:37.872307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:37009 (local address 127.5.46.126:41199)
0928 17:56:37.872381 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:37.872384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:37.872571 (+ 187us) server_negotiation.cc:407] Connection header received
0928 17:56:37.873143 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:37.873144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:37.873157 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:37.873220 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:37.874247 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.874769 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.875830 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.876076 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.876163 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:37.878879 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:37.878881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:37.878881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:37.878888 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:37.880749 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:37.881256 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:37.881257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:37.881258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:37.881312 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:37.881589 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:37.881590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:37.881591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:37.881722 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:37.881798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:37.882138 (+ 340us) server_negotiation.cc:299] Negotiation successful
0928 17:56:37.882242 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 17:56:37.882530 9577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:37.872184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41199 (local address 127.5.46.65:37009)
0928 17:56:37.872423 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:37.872482 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:37.873020 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:37.873300 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:37.873304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:37.873592 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:37.874125 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:37.874131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.874873 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.874874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:37.875722 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:37.875724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:37.876174 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:37.876175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:37.876321 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:37.876874 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:37.876882 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:37.878761 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:37.880854 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:37.880861 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:37.880863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:37.881108 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:37.881388 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:37.881389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:37.881389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:37.881519 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:37.881799 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:37.881806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:37.882000 (+ 194us) client_negotiation.cc:769] Sending connection context
0928 17:56:37.882107 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 17:56:37.882283 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":114,"threads_started":1}
I20260928 17:56:37.883044 9575 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41199
I20260928 17:56:37.883140 9575 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:37.883337 9575 heartbeater.cc:511] Master 127.5.46.126:41199 requested a full tablet report, sending...
I20260928 17:56:37.883935 9385 ts_manager.cc:194] Registered new tserver with Master: 00128c96108244ad9c0e58a83e446df2 (127.5.46.65:36439)
I20260928 17:56:37.884570 9385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:37009
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:37.908284 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:37.992208 9580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:37.992456 9580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:37.992499 9580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:37.993710 9580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:37.993784 9580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:37.993836 9580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:37.995404 9580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:37.995685 9580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:37.995973 9580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:38.001096 9580 server_base.cc:1034] running on GCE node
W20260928 17:56:38.001065 9588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.001072 9586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.001065 9585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.001463 9580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.001681 9580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.002825 9580 hybrid_clock.cc:648] HybridClock initialized: now 1790618198002813 us; error 27 us; skew 500 ppm
Sep 28 17:56:38 dist-test-slave-z2t3 krb5kdc[9341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618198, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:38.005510 9580 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:38.005827 9580 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.005872 9580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.005959 9580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.006995 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "79ba4cc118724eda9c4f3348e1600c7a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "be5ac4e75dd603bf6e559286340ef6e0"
server_key_iv: "75b34fc23294358f897b7682fa81cd2a"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.007314 9580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "79ba4cc118724eda9c4f3348e1600c7a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "be5ac4e75dd603bf6e559286340ef6e0"
server_key_iv: "75b34fc23294358f897b7682fa81cd2a"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.008621 9580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:38.009418 9594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.009653 9580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:38.009763 9580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "79ba4cc118724eda9c4f3348e1600c7a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "be5ac4e75dd603bf6e559286340ef6e0"
server_key_iv: "75b34fc23294358f897b7682fa81cd2a"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.009846 9580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.026041 9580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.028585 9580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.028759 9580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.029017 9580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:38.029350 9580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:38.029403 9580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.029448 9580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:38.029479 9580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.035418 9580 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:45521
I20260928 17:56:38.035494 9707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:45521 every 8 connection(s)
I20260928 17:56:38.035800 9580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:38 dist-test-slave-z2t3 krb5kdc[9341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618198, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:38.043751 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9580
I20260928 17:56:38.043943 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:38.044142 5305 external_mini_cluster.cc:1428] Setting key 9470eecd77fc2995447fb8ac1e24dcca
I20260928 17:56:38.046712 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.036604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:56619 (local address 127.5.46.126:41199)
0928 17:56:38.036677 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.036679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.036893 (+ 214us) server_negotiation.cc:407] Connection header received
0928 17:56:38.037420 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.037421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.037432 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.037499 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.038529 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.039037 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.040334 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.040543 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.040616 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.043107 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.043109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.043109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:38.043117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.045201 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.045630 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.045631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.045631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.045700 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.045953 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.045954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.045954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.046111 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.046181 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.046573 (+ 392us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.046643 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 17:56:38.047039 9710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.036576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41199 (local address 127.5.46.66:56619)
0928 17:56:38.036751 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.036795 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.037325 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.037578 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.037582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.037880 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.038399 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.038404 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.039156 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.039157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.040249 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.040252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.040629 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.040630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.040740 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.041387 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:38.041397 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.042990 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.045298 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:38.045306 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:38.045308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:38.045553 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:38.045779 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:38.045780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:38.045780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:38.045886 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:38.046181 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.046211 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.046416 (+ 205us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.046544 (+ 128us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.046754 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260928 17:56:38.047610 9708 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41199
I20260928 17:56:38.047740 9708 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:38.047937 9708 heartbeater.cc:511] Master 127.5.46.126:41199 requested a full tablet report, sending...
I20260928 17:56:38.048385 9385 ts_manager.cc:194] Registered new tserver with Master: 79ba4cc118724eda9c4f3348e1600c7a (127.5.46.66:45521)
I20260928 17:56:38.048784 9385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:56619
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:38.072366 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:38.154951 9713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:38.155213 9713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:38.155256 9713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:38.156504 9713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:38.156575 9713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:38.156625 9713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:38.158207 9713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41199
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:38.158491 9713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:38.158732 9713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:38.163942 9719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.163916 9721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.163934 9718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.164376 9713 server_base.cc:1034] running on GCE node
I20260928 17:56:38.164661 9713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.164947 9713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.166115 9713 hybrid_clock.cc:648] HybridClock initialized: now 1790618198166098 us; error 35 us; skew 500 ppm
Sep 28 17:56:38 dist-test-slave-z2t3 krb5kdc[9341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618198, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:38.169097 9713 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:38.169575 9713 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.169649 9713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.169795 9713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.170890 9713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "267494da51f6456bb00d6bf6eb81bcf0"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "a090b01727a1112a7bba94cb8f7e0289"
server_key_iv: "264f9082766b97a0087eb8184dae5320"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.171286 9713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "267494da51f6456bb00d6bf6eb81bcf0"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "a090b01727a1112a7bba94cb8f7e0289"
server_key_iv: "264f9082766b97a0087eb8184dae5320"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.172755 9713 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:38.173727 9727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.173946 9713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:38.174059 9713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "267494da51f6456bb00d6bf6eb81bcf0"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "a090b01727a1112a7bba94cb8f7e0289"
server_key_iv: "264f9082766b97a0087eb8184dae5320"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.174139 9713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.198016 9713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.199987 9713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.200145 9713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.200393 9713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:38.200728 9713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:38.200786 9713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.200830 9713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:38.200856 9713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.206570 9713 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:43789
I20260928 17:56:38.206604 9840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:43789 every 8 connection(s)
I20260928 17:56:38.206889 9713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:38.207559 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9713
I20260928 17:56:38.207651 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:38.207898 5305 external_mini_cluster.cc:1428] Setting key 8aba9a3d0d8b3b005190bee1a55428a3
Sep 28 17:56:38 dist-test-slave-z2t3 krb5kdc[9341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618198, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:38.216890 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.207729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:33497 (local address 127.5.46.126:41199)
0928 17:56:38.207789 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.207791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.208063 (+ 272us) server_negotiation.cc:407] Connection header received
0928 17:56:38.208654 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.208655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.208664 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.208701 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.209742 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.210316 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.211350 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.211550 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.211612 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.214085 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.214087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.214087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:38.214095 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.215631 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.216081 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.216082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.216082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.216135 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.216354 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.216354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.216354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.216458 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.216506 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.216759 (+ 253us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.216822 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 17:56:38.217124 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.207665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41199 (local address 127.5.46.67:33497)
0928 17:56:38.207896 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.207951 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.208541 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.208803 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.208807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.209128 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.209626 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.209632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.210432 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.210433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.211239 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.211241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.211627 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.211628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.211732 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.212398 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:38.212407 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.213974 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.215715 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:38.215722 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:38.215724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:38.216017 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:38.216195 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:38.216195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:38.216196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:38.216307 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:38.216509 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.216513 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.216639 (+ 126us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.216719 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.216849 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":86,"threads_started":1}
I20260928 17:56:38.217649 9841 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41199
I20260928 17:56:38.217756 9841 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:38.217932 9841 heartbeater.cc:511] Master 127.5.46.126:41199 requested a full tablet report, sending...
I20260928 17:56:38.218432 9385 ts_manager.cc:194] Registered new tserver with Master: 267494da51f6456bb00d6bf6eb81bcf0 (127.5.46.67:43789)
I20260928 17:56:38.218794 9385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:33497
I20260928 17:56:38.219714 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:56:38.227095 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.220943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.5.46.126:41199)
0928 17:56:38.221013 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.221016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.221130 (+ 114us) server_negotiation.cc:407] Connection header received
0928 17:56:38.221166 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.221167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.221176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.221225 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.221802 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.222296 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.223040 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.223224 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.223289 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.224608 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.224610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.224610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:38.224618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.225953 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.226366 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.226367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.226367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.226420 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:38.226649 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:38.226650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:38.226650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:38.226798 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.226873 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.226967 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.227029 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 17:56:38.229202 9385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:38.230875 9385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:38.237972 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43789 (local address 127.0.0.1:58686)
0928 17:56:38.234170 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.234179 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.234225 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.234407 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.234410 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.234412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:38.234488 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.234491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235364 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.235366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.236616 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.236618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237630 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.237723 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237740 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.237817 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.237877 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":113,"threads_started":1}
I20260928 17:56:38.238046 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44692 (local address 127.5.46.65:36439)
0928 17:56:38.233949 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.233952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.234055 (+ 103us) server_negotiation.cc:407] Connection header received
0928 17:56:38.234376 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.234380 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.234395 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.234473 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:38.234607 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235715 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237063 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237682 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237782 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237848 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.237926 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.237961 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260928 17:56:38.238281 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.5.46.66:45521)
0928 17:56:38.233853 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.233858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.234055 (+ 197us) server_negotiation.cc:407] Connection header received
0928 17:56:38.234160 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.234163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.234178 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.234258 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:38.234682 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235754 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237014 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237747 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237817 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237877 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.238161 (+ 284us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.238205 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260928 17:56:38.238473 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.5.46.67:43789)
0928 17:56:38.233985 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.233990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.234224 (+ 234us) server_negotiation.cc:407] Connection header received
0928 17:56:38.234290 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.234293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.234316 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.234418 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:38.234548 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235251 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.236704 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237459 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237530 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237593 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.238226 (+ 633us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.238271 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":73,"threads_started":1}
I20260928 17:56:38.238087 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36439 (local address 127.0.0.1:44692)
0928 17:56:38.233938 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.233982 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.234065 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.234470 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.234471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.234473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:38.234529 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.234532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235837 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.235838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.236972 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.236974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237786 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.237848 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237853 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.237911 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.237953 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":82,"threads_started":1}
I20260928 17:56:38.238197 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.233499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:45521 (local address 127.0.0.1:50428)
0928 17:56:38.233956 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.233987 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.234064 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.234559 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.234560 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.234562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:38.234599 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.234600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.235914 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.235915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.236917 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.236920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.237911 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.237912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.237978 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.237982 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.238038 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.238080 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":22,"thread_start_us":80,"threads_started":1}
I20260928 17:56:38.239432 9775 tablet_service.cc:1467] Processing CreateTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 (DEFAULT_TABLE table=test-table [id=4603d182095146fdb291bf52a92dffa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.239552 9509 tablet_service.cc:1467] Processing CreateTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 (DEFAULT_TABLE table=test-table [id=4603d182095146fdb291bf52a92dffa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.239591 9642 tablet_service.cc:1467] Processing CreateTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 (DEFAULT_TABLE table=test-table [id=4603d182095146fdb291bf52a92dffa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.239774 9775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7029fec3b20478f996d9e40f6f1ef38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.239902 9509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7029fec3b20478f996d9e40f6f1ef38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.239902 9642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7029fec3b20478f996d9e40f6f1ef38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.242858 9858 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Bootstrap starting.
I20260928 17:56:38.243458 9859 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Bootstrap starting.
I20260928 17:56:38.243515 9858 tablet_bootstrap.cc:654] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.244184 9859 tablet_bootstrap.cc:654] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.244202 9858 log.cc:824] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.244205 9860 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: Bootstrap starting.
I20260928 17:56:38.244760 9859 log.cc:824] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.245061 9860 tablet_bootstrap.cc:654] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.245446 9858 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: No bootstrap required, opened a new log
I20260928 17:56:38.245520 9858 ts_tablet_manager.cc:1397] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:56:38.245679 9859 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: No bootstrap required, opened a new log
I20260928 17:56:38.245743 9859 ts_tablet_manager.cc:1397] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 17:56:38.245707 9860 log.cc:824] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.246467 9860 tablet_bootstrap.cc:492] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: No bootstrap required, opened a new log
I20260928 17:56:38.246555 9860 ts_tablet_manager.cc:1397] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:38.247586 9858 raft_consensus.cc:348] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.247779 9858 raft_consensus.cc:374] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.247715 9859 raft_consensus.cc:348] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.247825 9858 raft_consensus.cc:729] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267494da51f6456bb00d6bf6eb81bcf0, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.247898 9859 raft_consensus.cc:374] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.247931 9859 raft_consensus.cc:729] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ba4cc118724eda9c4f3348e1600c7a, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.248006 9858 consensus_queue.cc:260] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.248041 9859 consensus_queue.cc:260] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.248210 9860 raft_consensus.cc:348] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.248320 9858 ts_tablet_manager.cc:1428] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:38.248327 9859 ts_tablet_manager.cc:1428] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260928 17:56:38.248385 9860 raft_consensus.cc:374] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.248437 9860 raft_consensus.cc:729] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00128c96108244ad9c0e58a83e446df2, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.248483 9708 heartbeater.cc:503] Master 127.5.46.126:41199 was elected leader, sending a full tablet report...
I20260928 17:56:38.248555 9860 consensus_queue.cc:260] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.248759 9841 heartbeater.cc:503] Master 127.5.46.126:41199 was elected leader, sending a full tablet report...
I20260928 17:56:38.248862 9860 ts_tablet_manager.cc:1428] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:56:38.249007 9575 heartbeater.cc:503] Master 127.5.46.126:41199 was elected leader, sending a full tablet report...
I20260928 17:56:38.259452 9865 raft_consensus.cc:482] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:38.259631 9865 raft_consensus.cc:504] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.260102 9865 leader_election.cc:288] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00128c96108244ad9c0e58a83e446df2 (127.5.46.65:36439), 79ba4cc118724eda9c4f3348e1600c7a (127.5.46.66:45521)
I20260928 17:56:38.263276 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.260409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58029 (local address 127.5.46.65:36439)
0928 17:56:38.260456 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.260458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.260610 (+ 152us) server_negotiation.cc:407] Connection header received
0928 17:56:38.260646 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.260646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.260659 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.260733 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:38.260895 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.261547 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.262518 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.262903 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.262987 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.262999 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.263159 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.263195 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 17:56:38.263296 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.260388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:35067 (local address 127.5.46.66:45521)
0928 17:56:38.260454 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.260456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.260467 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:38.260537 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.260538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.260550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.260638 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:38.260803 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.261445 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.262517 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.262903 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.262999 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.263009 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.263179 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.263220 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:56:38.263334 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.260340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:45521 (local address 127.5.46.67:35067)
0928 17:56:38.260417 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.260429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.260477 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.260644 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.260645 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.260647 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:38.260729 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.260731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.261544 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.261545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.262416 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.262418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.263000 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.263001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.263100 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.263104 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.263174 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.263224 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":3}
I20260928 17:56:38.263547 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.260354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36439 (local address 127.5.46.67:58029)
0928 17:56:38.260533 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.260541 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.260596 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.260735 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.260736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.260738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:38.260816 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.260820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.261612 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.261613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.262421 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:38.262423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.263000 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.263002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:38.263077 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.263086 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.263154 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.263196 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":25,"thread_start_us":118,"threads_started":1}
I20260928 17:56:38.263890 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7029fec3b20478f996d9e40f6f1ef38" candidate_uuid: "267494da51f6456bb00d6bf6eb81bcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00128c96108244ad9c0e58a83e446df2" is_pre_election: true
I20260928 17:56:38.264016 9662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7029fec3b20478f996d9e40f6f1ef38" candidate_uuid: "267494da51f6456bb00d6bf6eb81bcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ba4cc118724eda9c4f3348e1600c7a" is_pre_election: true
I20260928 17:56:38.264068 9529 raft_consensus.cc:2454] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267494da51f6456bb00d6bf6eb81bcf0 in term 0.
I20260928 17:56:38.264140 9662 raft_consensus.cc:2454] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 267494da51f6456bb00d6bf6eb81bcf0 in term 0.
I20260928 17:56:38.264318 9730 leader_election.cc:302] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00128c96108244ad9c0e58a83e446df2, 267494da51f6456bb00d6bf6eb81bcf0; no voters:
I20260928 17:56:38.264490 9865 raft_consensus.cc:2781] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:38.264575 9865 raft_consensus.cc:482] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:38.264611 9865 raft_consensus.cc:3037] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.265378 9865 raft_consensus.cc:504] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.265550 9865 leader_election.cc:288] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [CANDIDATE]: Term 1 election: Requested vote from peers 00128c96108244ad9c0e58a83e446df2 (127.5.46.65:36439), 79ba4cc118724eda9c4f3348e1600c7a (127.5.46.66:45521)
I20260928 17:56:38.265705 9662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7029fec3b20478f996d9e40f6f1ef38" candidate_uuid: "267494da51f6456bb00d6bf6eb81bcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79ba4cc118724eda9c4f3348e1600c7a"
I20260928 17:56:38.265805 9662 raft_consensus.cc:3037] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.266552 9662 raft_consensus.cc:2454] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267494da51f6456bb00d6bf6eb81bcf0 in term 1.
I20260928 17:56:38.266677 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7029fec3b20478f996d9e40f6f1ef38" candidate_uuid: "267494da51f6456bb00d6bf6eb81bcf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00128c96108244ad9c0e58a83e446df2"
I20260928 17:56:38.266743 9731 leader_election.cc:302] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 267494da51f6456bb00d6bf6eb81bcf0, 79ba4cc118724eda9c4f3348e1600c7a; no voters:
I20260928 17:56:38.266772 9529 raft_consensus.cc:3037] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.266834 9865 raft_consensus.cc:2781] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:38.267022 9865 raft_consensus.cc:686] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 1 LEADER]: Becoming Leader. State: Replica: 267494da51f6456bb00d6bf6eb81bcf0, State: Running, Role: LEADER
I20260928 17:56:38.267134 9865 consensus_queue.cc:237] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } }
I20260928 17:56:38.267676 9529 raft_consensus.cc:2454] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 267494da51f6456bb00d6bf6eb81bcf0 in term 1.
I20260928 17:56:38.268112 9385 catalog_manager.cc:5495] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 267494da51f6456bb00d6bf6eb81bcf0 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "267494da51f6456bb00d6bf6eb81bcf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "267494da51f6456bb00d6bf6eb81bcf0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43789 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:38.273494 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.271169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58702 (local address 127.5.46.67:43789)
0928 17:56:38.271281 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.271283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.271292 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:38.271331 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.271332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.271348 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.271410 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:38.271566 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.272054 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.272817 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:38.272995 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:38.273050 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:38.273188 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:38.273298 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:38.273365 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.273396 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.273413 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260928 17:56:38.275321 9529 raft_consensus.cc:1261] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 1 FOLLOWER]: Refusing update from remote peer 267494da51f6456bb00d6bf6eb81bcf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:38.275427 9662 raft_consensus.cc:1261] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 1 FOLLOWER]: Refusing update from remote peer 267494da51f6456bb00d6bf6eb81bcf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:38.275688 9868 consensus_queue.cc:1035] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00128c96108244ad9c0e58a83e446df2" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:38.275830 9865 consensus_queue.cc:1035] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79ba4cc118724eda9c4f3348e1600c7a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:38.278404 9871 mvcc.cc:205] Tried to move back new op lower bound from 7334372140132921344 to 7334372140103290880. Current Snapshot: MvccSnapshot[applied={T|T < 7334372140132921344}]
I20260928 17:56:38.278910 9873 mvcc.cc:205] Tried to move back new op lower bound from 7334372140132921344 to 7334372140103290880. Current Snapshot: MvccSnapshot[applied={T|T < 7334372140132921344 or (T in {7334372140132921344})}]
I20260928 17:56:38.278990 9872 mvcc.cc:205] Tried to move back new op lower bound from 7334372140132921344 to 7334372140103290880. Current Snapshot: MvccSnapshot[applied={T|T < 7334372140132921344}]
I20260928 17:56:38.281118 9384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:38.281229 9384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:38.282073 9384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3ebf290043a444c9c38b427c6d26ed1: Sending DeleteTablet for 3 replicas of tablet b7029fec3b20478f996d9e40f6f1ef38
I20260928 17:56:38.282444 9509 tablet_service.cc:1514] Processing DeleteTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44692
I20260928 17:56:38.282461 9642 tablet_service.cc:1514] Processing DeleteTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:50428
I20260928 17:56:38.282670 9880 tablet_replica.cc:330] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2: stopping tablet replica
I20260928 17:56:38.282775 9880 raft_consensus.cc:2229] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:38.282855 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9447
I20260928 17:56:38.282888 9880 raft_consensus.cc:2258] T b7029fec3b20478f996d9e40f6f1ef38 P 00128c96108244ad9c0e58a83e446df2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:38.282938 9775 tablet_service.cc:1514] Processing DeleteTablet for tablet b7029fec3b20478f996d9e40f6f1ef38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:58686
I20260928 17:56:38.283227 9881 tablet_replica.cc:330] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: stopping tablet replica
I20260928 17:56:38.283365 9881 raft_consensus.cc:2229] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:38.283512 9881 raft_consensus.cc:2258] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:38.283792 9882 tablet_replica.cc:330] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: stopping tablet replica
I20260928 17:56:38.283950 9882 raft_consensus.cc:2229] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:38.284097 9881 ts_tablet_manager.cc:1905] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:38.284113 9882 raft_consensus.cc:2258] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:38.284565 9882 ts_tablet_manager.cc:1905] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:38.286185 9882 ts_tablet_manager.cc:1918] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:38.286217 9881 ts_tablet_manager.cc:1918] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:38.286291 9881 log.cc:1196] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/b7029fec3b20478f996d9e40f6f1ef38
I20260928 17:56:38.286386 9882 log.cc:1196] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/b7029fec3b20478f996d9e40f6f1ef38
I20260928 17:56:38.286581 9881 ts_tablet_manager.cc:1939] T b7029fec3b20478f996d9e40f6f1ef38 P 79ba4cc118724eda9c4f3348e1600c7a: Deleting consensus metadata
I20260928 17:56:38.286787 9882 ts_tablet_manager.cc:1939] T b7029fec3b20478f996d9e40f6f1ef38 P 267494da51f6456bb00d6bf6eb81bcf0: Deleting consensus metadata
I20260928 17:56:38.287062 9372 catalog_manager.cc:4841] TS 79ba4cc118724eda9c4f3348e1600c7a (127.5.46.66:45521): tablet b7029fec3b20478f996d9e40f6f1ef38 (table test-table [id=4603d182095146fdb291bf52a92dffa1]) successfully deleted
I20260928 17:56:38.287442 9370 catalog_manager.cc:4841] TS 267494da51f6456bb00d6bf6eb81bcf0 (127.5.46.67:43789): tablet b7029fec3b20478f996d9e40f6f1ef38 (table test-table [id=4603d182095146fdb291bf52a92dffa1]) successfully deleted
W20260928 17:56:38.289930 9371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:36439 (error 108)
I20260928 17:56:38.290300 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9580
I20260928 17:56:38.290458 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.290340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36439 (local address 127.0.0.1:44704)
0928 17:56:38.290392 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.290426 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36439: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 17:56:38.291664 9371 catalog_manager.cc:4584] TS 00128c96108244ad9c0e58a83e446df2 (127.5.46.65:36439): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7029fec3b20478f996d9e40f6f1ef38: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36439: connect: Connection refused (error 111)
I20260928 17:56:38.294982 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9713
I20260928 17:56:38.299121 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9354
2026-09-28T17:56:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1927 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T17:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:38Z Disabled control of system clock
I20260928 17:56:38.327203 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35081
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:36861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:35081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 17:56:38.408915 9887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:38.409140 9887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:38.409162 9887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:38.410274 9887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:38.410319 9887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:38.410332 9887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:38.410344 9887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:38.410356 9887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:38.411723 9887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:35081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:38.411993 9887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:38.412251 9887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:38.417210 9893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.417207 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.417308 9887 server_base.cc:1034] running on GCE node
W20260928 17:56:38.417207 9895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.417745 9887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.418097 9887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.419241 9887 hybrid_clock.cc:648] HybridClock initialized: now 1790618198419227 us; error 33 us; skew 500 ppm
I20260928 17:56:38.420518 9887 webserver.cc:460] Webserver started at http://127.5.46.126:33511/ using document root <none> and password file <none>
I20260928 17:56:38.420760 9887 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.420819 9887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.420981 9887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.422147 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "7376f948754c46ce9fc992ec7e36a5fa"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "d2b92ea387ad0377ba40126aca59c356"
server_key_iv: "fbb4fb7ed33d978919a5249d3b461fbe"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.422508 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "7376f948754c46ce9fc992ec7e36a5fa"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "d2b92ea387ad0377ba40126aca59c356"
server_key_iv: "fbb4fb7ed33d978919a5249d3b461fbe"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.423944 9887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:38.424721 9901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.424937 9887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:38.425011 9887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "7376f948754c46ce9fc992ec7e36a5fa"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "d2b92ea387ad0377ba40126aca59c356"
server_key_iv: "fbb4fb7ed33d978919a5249d3b461fbe"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.425107 9887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.428658 9887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.428953 9887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.429090 9887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.433298 9887 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:35081
I20260928 17:56:38.433380 9953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:35081 every 8 connection(s)
I20260928 17:56:38.433678 9887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:38.434371 9954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.437321 9954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: Bootstrap starting.
I20260928 17:56:38.437932 9954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.438262 9954 log.cc:824] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.438971 9954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: No bootstrap required, opened a new log
I20260928 17:56:38.440688 9954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } }
I20260928 17:56:38.440832 9954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.440855 9954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7376f948754c46ce9fc992ec7e36a5fa, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.441025 9954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } }
I20260928 17:56:38.441089 9954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:38.441133 9954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:38.441184 9954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.441735 9954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } }
I20260928 17:56:38.441772 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9887
I20260928 17:56:38.441870 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:38.441854 9954 leader_election.cc:302] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7376f948754c46ce9fc992ec7e36a5fa; no voters:
I20260928 17:56:38.442063 5305 external_mini_cluster.cc:1428] Setting key f8930489ad87295d906a3840e073e97c
I20260928 17:56:38.442075 9954 leader_election.cc:288] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:38.442111 9957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:38.442234 9957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [term 1 LEADER]: Becoming Leader. State: Replica: 7376f948754c46ce9fc992ec7e36a5fa, State: Running, Role: LEADER
I20260928 17:56:38.442313 9957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } }
I20260928 17:56:38.442344 9954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:38.442981 9958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7376f948754c46ce9fc992ec7e36a5fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } } }
I20260928 17:56:38.443074 9958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:38.443238 9959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7376f948754c46ce9fc992ec7e36a5fa. Latest consensus state: current_term: 1 leader_uuid: "7376f948754c46ce9fc992ec7e36a5fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7376f948754c46ce9fc992ec7e36a5fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35081 } } }
I20260928 17:56:38.443291 9959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:38.443408 9968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:38.443940 9968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:38.445895 9968 catalog_manager.cc:1347] Generated new cluster ID: 0e17df5d75044e0fb01e62209c713eaa
I20260928 17:56:38.445952 9968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:38.446278 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.442803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48504 (local address 127.5.46.126:35081)
0928 17:56:38.443013 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.443023 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.444746 (+ 1723us) server_negotiation.cc:407] Connection header received
0928 17:56:38.444878 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.444886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.444942 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.445104 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.445783 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.445786 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.445786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.445789 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.445854 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.445870 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.445934 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.446008 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.446065 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260928 17:56:38.450891 9968 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:38.451049 9968 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:38.453547 9968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: Generated new TSK 0
I20260928 17:56:38.453714 9968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:56:38.455935 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--builtin_ntp_servers=127.5.46.84:36861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 17:56:38.556602 9978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:38.556867 9978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:38.556911 9978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:38.558199 9978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:38.558272 9978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:38.558318 9978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:38.559772 9978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:38.560084 9978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:38.560410 9978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:38.567265 9983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.567283 9984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.567283 9986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.567493 9978 server_base.cc:1034] running on GCE node
I20260928 17:56:38.567701 9978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.568015 9978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.569167 9978 hybrid_clock.cc:648] HybridClock initialized: now 1790618198569144 us; error 41 us; skew 500 ppm
I20260928 17:56:38.570556 9978 webserver.cc:460] Webserver started at http://127.5.46.65:36391/ using document root <none> and password file <none>
I20260928 17:56:38.570863 9978 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.570943 9978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.571097 9978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.572285 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "dd36967de23f4453b364cb4eccfa851a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "041749a0899fe346762d4c88c2360a9c"
server_key_iv: "2fe3a71812e9f77ed79df1f0f8575bb7"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.572646 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "dd36967de23f4453b364cb4eccfa851a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "041749a0899fe346762d4c88c2360a9c"
server_key_iv: "2fe3a71812e9f77ed79df1f0f8575bb7"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.574131 9978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 17:56:38.575089 9992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.575377 9978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:38.575513 9978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "dd36967de23f4453b364cb4eccfa851a"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "041749a0899fe346762d4c88c2360a9c"
server_key_iv: "2fe3a71812e9f77ed79df1f0f8575bb7"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.575609 9978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.579422 9978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.579838 9978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.580072 9978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.580305 9978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:38.580679 9978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:38.580740 9978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.580788 9978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:38.580819 9978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.587580 9978 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33801
I20260928 17:56:38.587625 10105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33801 every 8 connection(s)
I20260928 17:56:38.588061 9978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:38.591588 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.588887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:56059 (local address 127.5.46.126:35081)
0928 17:56:38.588951 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.588953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.589258 (+ 305us) server_negotiation.cc:407] Connection header received
0928 17:56:38.589833 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.589834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.589848 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.589904 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.591256 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.591257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.591258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.591264 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.591285 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.591291 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.591376 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.591486 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.591506 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:56:38.591929 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.588863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35081 (local address 127.5.46.65:56059)
0928 17:56:38.589059 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.589095 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.589686 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.589983 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.589987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.589995 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.591062 (+ 1067us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.591062 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.591095 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.591128 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.591386 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.591393 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.591396 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.591499 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.591571 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260928 17:56:38.591974 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 9978
I20260928 17:56:38.592072 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:38.592247 5305 external_mini_cluster.cc:1428] Setting key 2e3d638aa3b5c96c5c0766a2e81c20b6
I20260928 17:56:38.592933 10106 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35081
I20260928 17:56:38.593056 10106 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:38.593324 10106 heartbeater.cc:511] Master 127.5.46.126:35081 requested a full tablet report, sending...
I20260928 17:56:38.593389 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--builtin_ntp_servers=127.5.46.84:36861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 17:56:38.594103 9918 ts_manager.cc:194] Registered new tserver with Master: dd36967de23f4453b364cb4eccfa851a (127.5.46.65:33801)
W20260928 17:56:38.680541 10109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:38.680759 10109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:38.680784 10109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:38.682041 10109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:38.682085 10109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:38.682111 10109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:38.683509 10109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:38.683779 10109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:38.684120 10109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:38.689239 10117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.689308 10115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.689474 10109 server_base.cc:1034] running on GCE node
W20260928 17:56:38.689246 10114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.689774 10109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.690023 10109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.691166 10109 hybrid_clock.cc:648] HybridClock initialized: now 1790618198691155 us; error 27 us; skew 500 ppm
I20260928 17:56:38.692543 10109 webserver.cc:460] Webserver started at http://127.5.46.66:40137/ using document root <none> and password file <none>
I20260928 17:56:38.692812 10109 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.692862 10109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.693004 10109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.694113 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "aa47b8a2275349acb0d5a7dfa602547e"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "b9c93087670e02cada167f560f8c5839"
server_key_iv: "ce5b5674971fd65bd8599fc830a007bd"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.694505 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "aa47b8a2275349acb0d5a7dfa602547e"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "b9c93087670e02cada167f560f8c5839"
server_key_iv: "ce5b5674971fd65bd8599fc830a007bd"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.695982 10109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 17:56:38.696831 10123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.697019 10109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:38.697129 10109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "aa47b8a2275349acb0d5a7dfa602547e"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "b9c93087670e02cada167f560f8c5839"
server_key_iv: "ce5b5674971fd65bd8599fc830a007bd"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.697208 10109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.700774 10109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.701064 10109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.701210 10109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.701427 10109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:38.701741 10109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:38.701797 10109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.701838 10109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:38.701865 10109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.707628 10109 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:36111
I20260928 17:56:38.707720 10236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:36111 every 8 connection(s)
I20260928 17:56:38.708089 10109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:38.709106 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10109
I20260928 17:56:38.709218 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:38.709385 5305 external_mini_cluster.cc:1428] Setting key 93e31aad4d2428e0f03c557c25a67213
I20260928 17:56:38.710423 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--builtin_ntp_servers=127.5.46.84:36861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260928 17:56:38.711684 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.708952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:47535 (local address 127.5.46.126:35081)
0928 17:56:38.709025 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.709028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.709228 (+ 200us) server_negotiation.cc:407] Connection header received
0928 17:56:38.709904 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.709905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.709912 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.709959 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.711415 (+ 1456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.711417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.711417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.711421 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.711439 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.711443 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.711508 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.711612 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.711626 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 17:56:38.711980 10239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.708920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35081 (local address 127.5.46.66:47535)
0928 17:56:38.709106 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.709148 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.709790 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.710037 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.710042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.710052 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.711212 (+ 1160us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.711213 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.711255 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.711293 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.711523 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.711528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.711531 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.711612 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.711680 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260928 17:56:38.712425 10237 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35081
I20260928 17:56:38.712517 10237 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:38.712685 10237 heartbeater.cc:511] Master 127.5.46.126:35081 requested a full tablet report, sending...
I20260928 17:56:38.713028 9918 ts_manager.cc:194] Registered new tserver with Master: aa47b8a2275349acb0d5a7dfa602547e (127.5.46.66:36111)
W20260928 17:56:38.795956 10240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:38.796216 10240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:38.796257 10240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:38.797477 10240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:38.797559 10240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:38.797603 10240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:38.799075 10240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:38.799392 10240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:38.799721 10240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:38.805042 10248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.805166 10240 server_base.cc:1034] running on GCE node
W20260928 17:56:38.805042 10245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:38.805266 10246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:38.805478 10240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:38.805799 10240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:38.806965 10240 hybrid_clock.cc:648] HybridClock initialized: now 1790618198806965 us; error 42 us; skew 500 ppm
I20260928 17:56:38.808292 10240 webserver.cc:460] Webserver started at http://127.5.46.67:44709/ using document root <none> and password file <none>
I20260928 17:56:38.808555 10240 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:38.808625 10240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:38.808779 10240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:38.809893 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "26f8c3f36aca44c48231b8f22a573edc"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "fa3efb72c4b3579e52dabd1827d90fc8"
server_key_iv: "b24d87b6360cbe6eaef437cd74816c60"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.810263 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "26f8c3f36aca44c48231b8f22a573edc"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "fa3efb72c4b3579e52dabd1827d90fc8"
server_key_iv: "b24d87b6360cbe6eaef437cd74816c60"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.811609 10240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 17:56:38.812423 10254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.812597 10240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.812691 10240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "26f8c3f36aca44c48231b8f22a573edc"
format_stamp: "Formatted at 2026-09-28 17:56:38 on dist-test-slave-z2t3"
server_key: "fa3efb72c4b3579e52dabd1827d90fc8"
server_key_iv: "b24d87b6360cbe6eaef437cd74816c60"
server_key_version: "encryptionkey@0"
I20260928 17:56:38.812772 10240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:38.816200 10240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:38.816478 10240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:38.816622 10240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:38.816829 10240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:38.817142 10240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:38.817201 10240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.817240 10240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:38.817268 10240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:38.822842 10240 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45733
I20260928 17:56:38.822918 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45733 every 8 connection(s)
I20260928 17:56:38.823242 10240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:38.826100 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10240
I20260928 17:56:38.826210 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:38.826411 5305 external_mini_cluster.cc:1428] Setting key d014d158ee997db478f097320df325e2
I20260928 17:56:38.826519 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.824047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:59323 (local address 127.5.46.126:35081)
0928 17:56:38.824122 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.824124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.824380 (+ 256us) server_negotiation.cc:407] Connection header received
0928 17:56:38.824834 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.824835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.824841 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.824905 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.826152 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.826153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.826154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.826158 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.826174 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.826178 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.826282 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.826428 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.826447 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 17:56:38.826730 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.824057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35081 (local address 127.5.46.67:59323)
0928 17:56:38.824253 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.824287 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.824730 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.824956 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.824959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.824968 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.825931 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.825932 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.825974 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.826015 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.826313 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.826321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.826325 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.826409 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.826486 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
I20260928 17:56:38.827180 10368 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35081
I20260928 17:56:38.827297 10368 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:38.827510 10368 heartbeater.cc:511] Master 127.5.46.126:35081 requested a full tablet report, sending...
I20260928 17:56:38.827824 9918 ts_manager.cc:194] Registered new tserver with Master: 26f8c3f36aca44c48231b8f22a573edc (127.5.46.67:45733)
I20260928 17:56:38.838584 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:56:38.841243 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.840017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48518 (local address 127.5.46.126:35081)
0928 17:56:38.840082 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.840084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.840211 (+ 127us) server_negotiation.cc:407] Connection header received
0928 17:56:38.840280 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.840281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.840287 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.840382 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.841004 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.841005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.841006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.841010 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.841025 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.841030 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.841096 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.841147 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.841163 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 17:56:38.842970 9918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:38.844743 9918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:38.849036 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:45733 (local address 127.0.0.1:60176)
0928 17:56:38.847252 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.847297 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.847358 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.847961 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.847968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.847976 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.848563 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.848564 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.848577 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.848596 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.848855 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.848861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.848866 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.848936 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.848950 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260928 17:56:38.849079 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.5.46.67:45733)
0928 17:56:38.847695 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.847701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.847713 (+ 12us) server_negotiation.cc:407] Connection header received
0928 17:56:38.847740 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.847742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.847752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.848209 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.848678 (+ 469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.848684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.848684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.848688 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.848719 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.848738 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.848853 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.848957 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.848974 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260928 17:56:38.849154 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:36111 (local address 127.0.0.1:37928)
0928 17:56:38.847272 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.847296 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.847389 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.848289 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.848290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.848292 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.848707 (+ 415us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.848707 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.848715 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.848729 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.848994 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.848995 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.848996 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.849041 (+ 45us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.849053 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":4,"thread_start_us":82,"threads_started":1}
I20260928 17:56:38.849375 10379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33801 (local address 127.0.0.1:39118)
0928 17:56:38.847608 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.847618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.847674 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.848104 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.848105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.848107 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.849006 (+ 899us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.849007 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.849012 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.849022 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.849249 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.849250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.849251 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.849306 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.849321 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":77,"threads_started":1}
I20260928 17:56:38.849445 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.5.46.65:33801)
0928 17:56:38.847967 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.847972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.847981 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:38.848004 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.848006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.848015 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.848086 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.849107 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.849115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.849115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.849119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.849154 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.849176 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.849240 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.849322 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.849348 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":103,"threads_started":1}
I20260928 17:56:38.849453 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.847187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37928 (local address 127.5.46.66:36111)
0928 17:56:38.847796 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.847801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.847812 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:38.847867 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.847869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.847878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.847953 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.848817 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.848825 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.848826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.848830 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.848862 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.848894 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.848968 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.849340 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.849356 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":149,"threads_started":1}
I20260928 17:56:38.850097 10040 tablet_service.cc:1467] Processing CreateTablet for tablet 52208602359e4b9eafabfa170fb39d18 (DEFAULT_TABLE table=test-table [id=1be98f174f2041b59a6ed018fa593437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.850112 10171 tablet_service.cc:1467] Processing CreateTablet for tablet 52208602359e4b9eafabfa170fb39d18 (DEFAULT_TABLE table=test-table [id=1be98f174f2041b59a6ed018fa593437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.850292 10302 tablet_service.cc:1467] Processing CreateTablet for tablet 52208602359e4b9eafabfa170fb39d18 (DEFAULT_TABLE table=test-table [id=1be98f174f2041b59a6ed018fa593437]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:38.850466 10171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52208602359e4b9eafabfa170fb39d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.850477 10040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52208602359e4b9eafabfa170fb39d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.850566 10302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52208602359e4b9eafabfa170fb39d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:38.853034 10382 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: Bootstrap starting.
I20260928 17:56:38.853770 10382 tablet_bootstrap.cc:654] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.853823 10383 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: Bootstrap starting.
I20260928 17:56:38.853823 10384 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Bootstrap starting.
I20260928 17:56:38.854094 10382 log.cc:824] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.854458 10384 tablet_bootstrap.cc:654] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.854460 10383 tablet_bootstrap.cc:654] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:38.854800 10384 log.cc:824] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.854800 10383 log.cc:824] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:38.854828 10382 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: No bootstrap required, opened a new log
I20260928 17:56:38.854918 10382 ts_tablet_manager.cc:1397] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:38.855612 10383 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: No bootstrap required, opened a new log
I20260928 17:56:38.855679 10383 ts_tablet_manager.cc:1397] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:38.856220 10384 tablet_bootstrap.cc:492] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: No bootstrap required, opened a new log
I20260928 17:56:38.856273 10384 ts_tablet_manager.cc:1397] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:38.857527 10382 raft_consensus.cc:348] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.857723 10382 raft_consensus.cc:374] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.857759 10382 raft_consensus.cc:729] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd36967de23f4453b364cb4eccfa851a, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.857918 10382 consensus_queue.cc:260] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.858067 10384 raft_consensus.cc:348] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.858194 10382 ts_tablet_manager.cc:1428] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 17:56:38.858242 10384 raft_consensus.cc:374] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.858250 10106 heartbeater.cc:503] Master 127.5.46.126:35081 was elected leader, sending a full tablet report...
I20260928 17:56:38.858299 10384 raft_consensus.cc:729] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f8c3f36aca44c48231b8f22a573edc, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.858299 10383 raft_consensus.cc:348] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.858440 10383 raft_consensus.cc:374] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:38.858412 10384 consensus_queue.cc:260] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.858464 10383 raft_consensus.cc:729] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa47b8a2275349acb0d5a7dfa602547e, State: Initialized, Role: FOLLOWER
I20260928 17:56:38.858568 10383 consensus_queue.cc:260] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.858651 10384 ts_tablet_manager.cc:1428] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:56:38.858808 10383 ts_tablet_manager.cc:1428] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:38.858810 10368 heartbeater.cc:503] Master 127.5.46.126:35081 was elected leader, sending a full tablet report...
I20260928 17:56:38.859097 10237 heartbeater.cc:503] Master 127.5.46.126:35081 was elected leader, sending a full tablet report...
I20260928 17:56:38.873281 10388 raft_consensus.cc:482] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:38.873442 10388 raft_consensus.cc:504] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.873780 10388 leader_election.cc:288] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 26f8c3f36aca44c48231b8f22a573edc (127.5.46.67:45733), aa47b8a2275349acb0d5a7dfa602547e (127.5.46.66:36111)
I20260928 17:56:38.875326 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.874077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:56019 (local address 127.5.46.67:45733)
0928 17:56:38.874163 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.874165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.874175 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:38.874227 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.874228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.874233 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.874357 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.874984 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.874985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.874986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.874997 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.875017 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.875022 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.875098 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.875231 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.875247 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 17:56:38.875325 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.874021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:45733 (local address 127.5.46.65:56019)
0928 17:56:38.874100 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.874114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.874166 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.874353 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.874354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.874357 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.874852 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.874853 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.874861 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.874877 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.875133 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.875134 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.875135 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.875230 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.875249 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260928 17:56:38.875428 10378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.874098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:39841 (local address 127.5.46.66:36111)
0928 17:56:38.874154 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.874156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.874380 (+ 224us) server_negotiation.cc:407] Connection header received
0928 17:56:38.874452 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.874453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.874458 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.874530 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.875140 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.875141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.875141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.875147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.875165 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.875169 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.875233 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.875338 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.875352 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 17:56:38.875451 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.874022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:36111 (local address 127.5.46.65:39841)
0928 17:56:38.874307 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.874316 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:38.874388 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:38.874534 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:38.874534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:38.874537 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:38.875014 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:56:38.875014 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:38.875021 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:56:38.875033 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:38.875238 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:38.875239 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:38.875240 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:38.875291 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 17:56:38.875303 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":64,"thread_start_us":113,"threads_started":1}
I20260928 17:56:38.875746 10191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52208602359e4b9eafabfa170fb39d18" candidate_uuid: "dd36967de23f4453b364cb4eccfa851a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa47b8a2275349acb0d5a7dfa602547e" is_pre_election: true
I20260928 17:56:38.875705 10322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52208602359e4b9eafabfa170fb39d18" candidate_uuid: "dd36967de23f4453b364cb4eccfa851a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f8c3f36aca44c48231b8f22a573edc" is_pre_election: true
I20260928 17:56:38.875936 10191 raft_consensus.cc:2454] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd36967de23f4453b364cb4eccfa851a in term 0.
I20260928 17:56:38.875952 10322 raft_consensus.cc:2454] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd36967de23f4453b364cb4eccfa851a in term 0.
I20260928 17:56:38.876204 9995 leader_election.cc:302] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa47b8a2275349acb0d5a7dfa602547e, dd36967de23f4453b364cb4eccfa851a; no voters:
I20260928 17:56:38.876339 10388 raft_consensus.cc:2781] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:38.876397 10388 raft_consensus.cc:482] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:38.876421 10388 raft_consensus.cc:3037] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.877121 10388 raft_consensus.cc:504] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.877291 10388 leader_election.cc:288] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [CANDIDATE]: Term 1 election: Requested vote from peers 26f8c3f36aca44c48231b8f22a573edc (127.5.46.67:45733), aa47b8a2275349acb0d5a7dfa602547e (127.5.46.66:36111)
I20260928 17:56:38.877435 10191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52208602359e4b9eafabfa170fb39d18" candidate_uuid: "dd36967de23f4453b364cb4eccfa851a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa47b8a2275349acb0d5a7dfa602547e"
I20260928 17:56:38.877482 10322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52208602359e4b9eafabfa170fb39d18" candidate_uuid: "dd36967de23f4453b364cb4eccfa851a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26f8c3f36aca44c48231b8f22a573edc"
I20260928 17:56:38.877524 10191 raft_consensus.cc:3037] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.877554 10322 raft_consensus.cc:3037] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:38.878444 10322 raft_consensus.cc:2454] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd36967de23f4453b364cb4eccfa851a in term 1.
I20260928 17:56:38.878444 10191 raft_consensus.cc:2454] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd36967de23f4453b364cb4eccfa851a in term 1.
I20260928 17:56:38.878602 9996 leader_election.cc:302] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f8c3f36aca44c48231b8f22a573edc, dd36967de23f4453b364cb4eccfa851a; no voters:
I20260928 17:56:38.878727 10388 raft_consensus.cc:2781] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:38.878899 10388 raft_consensus.cc:686] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [term 1 LEADER]: Becoming Leader. State: Replica: dd36967de23f4453b364cb4eccfa851a, State: Running, Role: LEADER
I20260928 17:56:38.879002 10388 consensus_queue.cc:237] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } }
I20260928 17:56:38.879897 9917 catalog_manager.cc:5495] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a reported cstate change: term changed from 0 to 1, leader changed from <none> to dd36967de23f4453b364cb4eccfa851a (127.5.46.65). New cstate: current_term: 1 leader_uuid: "dd36967de23f4453b364cb4eccfa851a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd36967de23f4453b364cb4eccfa851a" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33801 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:38.889796 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.888796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.5.46.65:33801)
0928 17:56:38.888845 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:38.888847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:38.888858 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:38.888930 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:38.888930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:38.888942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:38.888996 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:38.889627 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:38.889628 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:38.889628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:38.889630 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:38.889641 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:38.889644 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:38.889683 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:38.889734 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 17:56:38.889743 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:56:38.891459 10191 raft_consensus.cc:1261] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 1 FOLLOWER]: Refusing update from remote peer dd36967de23f4453b364cb4eccfa851a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:38.891459 10322 raft_consensus.cc:1261] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 1 FOLLOWER]: Refusing update from remote peer dd36967de23f4453b364cb4eccfa851a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:38.891889 10388 consensus_queue.cc:1035] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f8c3f36aca44c48231b8f22a573edc" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:38.892037 10392 consensus_queue.cc:1035] T 52208602359e4b9eafabfa170fb39d18 P dd36967de23f4453b364cb4eccfa851a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa47b8a2275349acb0d5a7dfa602547e" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:38.894963 10396 mvcc.cc:205] Tried to move back new op lower bound from 7334372142656921600 to 7334372142609133568. Current Snapshot: MvccSnapshot[applied={T|T < 7334372142656921600}]
I20260928 17:56:38.895800 10397 mvcc.cc:205] Tried to move back new op lower bound from 7334372142656921600 to 7334372142609133568. Current Snapshot: MvccSnapshot[applied={T|T < 7334372142656921600 or (T in {7334372142656921600})}]
I20260928 17:56:38.897164 9917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:38.897264 9917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:38.898056 9917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7376f948754c46ce9fc992ec7e36a5fa: Sending DeleteTablet for 3 replicas of tablet 52208602359e4b9eafabfa170fb39d18
I20260928 17:56:38.898522 10040 tablet_service.cc:1514] Processing DeleteTablet for tablet 52208602359e4b9eafabfa170fb39d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='slave'} at 127.0.0.1:39118
I20260928 17:56:38.898756 10171 tablet_service.cc:1514] Processing DeleteTablet for tablet 52208602359e4b9eafabfa170fb39d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='slave'} at 127.0.0.1:37928
I20260928 17:56:38.898972 10405 tablet_replica.cc:330] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e: stopping tablet replica
I20260928 17:56:38.899004 10302 tablet_service.cc:1514] Processing DeleteTablet for tablet 52208602359e4b9eafabfa170fb39d18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:38 UTC) from {username='slave'} at 127.0.0.1:60176
I20260928 17:56:38.899096 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9978
I20260928 17:56:38.900141 10406 tablet_replica.cc:330] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: stopping tablet replica
I20260928 17:56:38.900269 10406 raft_consensus.cc:2229] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:38.900400 10406 raft_consensus.cc:2258] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:38.900897 10406 ts_tablet_manager.cc:1905] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:38.902760 10406 ts_tablet_manager.cc:1918] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:38.902834 10406 log.cc:1196] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/52208602359e4b9eafabfa170fb39d18
I20260928 17:56:38.903036 10406 ts_tablet_manager.cc:1939] T 52208602359e4b9eafabfa170fb39d18 P 26f8c3f36aca44c48231b8f22a573edc: Deleting consensus metadata
I20260928 17:56:38.903457 9905 catalog_manager.cc:4841] TS 26f8c3f36aca44c48231b8f22a573edc (127.5.46.67:45733): tablet 52208602359e4b9eafabfa170fb39d18 (table test-table [id=1be98f174f2041b59a6ed018fa593437]) successfully deleted
W20260928 17:56:38.903669 9905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:33801 (error 108)
I20260928 17:56:38.904103 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10109
I20260928 17:56:38.904117 10405 raft_consensus.cc:2229] T 52208602359e4b9eafabfa170fb39d18 P aa47b8a2275349acb0d5a7dfa602547e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:38.904206 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.904053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33801 (local address 127.0.0.1:39134)
0928 17:56:38.904119 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.904172 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 17:56:38.908586 9905 catalog_manager.cc:4584] TS dd36967de23f4453b364cb4eccfa851a (127.5.46.65:33801): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52208602359e4b9eafabfa170fb39d18: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33801: connect: Connection refused (error 111)
I20260928 17:56:38.908756 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10240
I20260928 17:56:38.908838 10376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:38.908715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:36111 (local address 127.0.0.1:37932)
0928 17:56:38.908775 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:38.908800 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.66:36111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 17:56:38.913324 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 9887
2026-09-28T17:56:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (613 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 17:56:38.933228 5305 mini_oidc.cc:152] Starting JWKS server
I20260928 17:56:38.934230 5305 webserver.cc:460] Webserver started at https://127.0.0.1:37785/ using document root <none> and password file <none>
I20260928 17:56:38.934315 5305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 17:56:38.934504 5305 webserver.cc:460] Webserver started at http://127.0.0.1:46359/ using document root <none> and password file <none>
2026-09-28T17:56:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:38Z Disabled control of system clock
I20260928 17:56:38.942693 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44435
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:35643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:44435
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37785/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/testchainca.cert with env {}
W20260928 17:56:39.024935 10413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:39.025158 10413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:39.025182 10413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:39.026448 10413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:39.026500 10413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:39.026513 10413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:39.026526 10413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:39.026538 10413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 17:56:39.026582 10413 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 17:56:39.026612 10413 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37785/jwks.json
I20260928 17:56:39.028297 10413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:35643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:44435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37785/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:39.028602 10413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:39.028855 10413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:39.034008 10419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.034024 10418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:39.034323 10413 server_base.cc:1034] running on GCE node
W20260928 17:56:39.034008 10421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:39.034650 10413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:39.034937 10413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:39.036123 10413 hybrid_clock.cc:648] HybridClock initialized: now 1790618199036097 us; error 35 us; skew 500 ppm
I20260928 17:56:39.037454 10413 webserver.cc:460] Webserver started at http://127.5.46.126:41275/ using document root <none> and password file <none>
I20260928 17:56:39.037725 10413 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:39.037799 10413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:39.037967 10413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:39.039091 10413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "b2f058bba9cc444f9f13df6aea837dd9"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "cb04109a9769a13b7a29ac1546a2a698"
server_key_iv: "d0560acd0ea4d23ef221299896b0331c"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.039475 10413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "b2f058bba9cc444f9f13df6aea837dd9"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "cb04109a9769a13b7a29ac1546a2a698"
server_key_iv: "d0560acd0ea4d23ef221299896b0331c"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.040931 10413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:39.041728 10427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:39.041944 10413 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:39.042066 10413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "b2f058bba9cc444f9f13df6aea837dd9"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "cb04109a9769a13b7a29ac1546a2a698"
server_key_iv: "d0560acd0ea4d23ef221299896b0331c"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.042142 10413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:39.063527 10413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:39.063957 10413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:39.064126 10413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:39.068105 10413 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:44435
I20260928 17:56:39.068212 10479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:44435 every 8 connection(s)
I20260928 17:56:39.068580 10413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:39.069643 10480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:39.072196 10480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: Bootstrap starting.
I20260928 17:56:39.072813 10480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:39.073071 10480 log.cc:824] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:39.073803 10480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: No bootstrap required, opened a new log
I20260928 17:56:39.075408 10480 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } }
I20260928 17:56:39.075560 10480 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:39.075584 10480 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2f058bba9cc444f9f13df6aea837dd9, State: Initialized, Role: FOLLOWER
I20260928 17:56:39.075742 10480 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } }
I20260928 17:56:39.075834 10480 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:39.075934 10480 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:39.075974 10480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:39.076486 10480 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } }
I20260928 17:56:39.076570 10480 leader_election.cc:302] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2f058bba9cc444f9f13df6aea837dd9; no voters:
I20260928 17:56:39.076731 10480 leader_election.cc:288] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:39.076768 10483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:39.076903 10483 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [term 1 LEADER]: Becoming Leader. State: Replica: b2f058bba9cc444f9f13df6aea837dd9, State: Running, Role: LEADER
I20260928 17:56:39.077009 10480 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:39.077239 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10413
I20260928 17:56:39.077335 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:39.077531 5305 external_mini_cluster.cc:1428] Setting key e12e3ab0bd438b115003863f6c888cb2
I20260928 17:56:39.076993 10483 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } }
I20260928 17:56:39.078485 10484 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2f058bba9cc444f9f13df6aea837dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } } }
I20260928 17:56:39.078763 10484 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.078583 10485 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2f058bba9cc444f9f13df6aea837dd9. Latest consensus state: current_term: 1 leader_uuid: "b2f058bba9cc444f9f13df6aea837dd9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2f058bba9cc444f9f13df6aea837dd9" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44435 } } }
I20260928 17:56:39.078881 10485 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9 [sys.catalog]: This master's current role is: LEADER
W20260928 17:56:39.079452 10499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:56:39.079512 10499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:56:39.079569 10500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:39.079734 10500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:39.082144 10500 catalog_manager.cc:1347] Generated new cluster ID: 33942a084b444e5b957a2d6ba473a103
I20260928 17:56:39.082211 10500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:39.084251 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.078265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.5.46.126:44435)
0928 17:56:39.078999 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.079008 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.079047 (+ 39us) server_negotiation.cc:407] Connection header received
0928 17:56:39.079178 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.079197 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.079277 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.079425 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:39.079662 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.080959 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.082092 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.082619 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.082769 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.083503 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:39.083507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:39.083508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:39.083693 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:39.083726 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:39.083738 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:39.083824 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:39.083939 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 17:56:39.084022 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":80,"threads_started":1}
I20260928 17:56:39.085150 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:56:39.120594 10500 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:39.121343 10500 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:39.126752 10500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2f058bba9cc444f9f13df6aea837dd9: Generated new TSK 0
I20260928 17:56:39.126996 10500 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 17:56:39.158303 10510 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 17:56:39.158432 10510 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260928 17:56:39.159116 10509 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:39.134853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44435 (local address 127.0.0.1:36322)
0928 17:56:39.135052 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:39.135063 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:39.135143 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:39.135348 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:39.135348 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:39.135350 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 17:56:39.135419 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.135421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.136203 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.136204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.137187 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.137191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.137578 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.137579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.137649 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.137675 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 17:56:39.158628 (+ 20953us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 17:56:39.158755 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.46.126:44435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":116,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.46.126:44435: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 17:56:39.159461 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.134909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.5.46.126:44435)
0928 17:56:39.134980 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.134982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.135156 (+ 174us) server_negotiation.cc:407] Connection header received
0928 17:56:39.135238 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.135239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.135253 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.135330 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0928 17:56:39.135489 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.136101 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.137297 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.137493 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.137558 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.137888 (+ 330us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 17:56:39.158442 (+ 20554us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 17:56:39.159390 (+ 948us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260928 17:56:39.159503 10496 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36322: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37785/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
I20260928 17:56:39.159514 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10413
2026-09-28T17:56:39Z chronyd exiting
I20260928 17:56:39.195802 5305 test_util.cc:175] -----------------------------------------------
I20260928 17:56:39.196003 5305 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790618177091767-5305-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (263 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 17:56:39.196882 5305 mini_oidc.cc:152] Starting JWKS server
I20260928 17:56:39.197677 5305 webserver.cc:460] Webserver started at https://127.0.0.1:38419/ using document root <none> and password file <none>
I20260928 17:56:39.197757 5305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 17:56:39.197944 5305 webserver.cc:460] Webserver started at http://127.0.0.1:41805/ using document root <none> and password file <none>
2026-09-28T17:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:39Z Disabled control of system clock
I20260928 17:56:39.206445 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39159
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:39159
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38419/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/testchainca.cert with env {}
W20260928 17:56:39.290602 10516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:39.290882 10516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:39.290925 10516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:39.292188 10516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:39.292297 10516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:39.292318 10516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:39.292339 10516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:39.292371 10516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 17:56:39.292395 10516 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 17:56:39.292424 10516 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38419/jwks.json
I20260928 17:56:39.294070 10516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:39159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38419/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:39.294390 10516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:39.294633 10516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:39.300161 10516 server_base.cc:1034] running on GCE node
W20260928 17:56:39.300172 10521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.300161 10524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.300161 10522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:39.300783 10516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:39.301110 10516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:39.302277 10516 hybrid_clock.cc:648] HybridClock initialized: now 1790618199302251 us; error 43 us; skew 500 ppm
I20260928 17:56:39.303593 10516 webserver.cc:460] Webserver started at http://127.5.46.126:37263/ using document root <none> and password file <none>
I20260928 17:56:39.303936 10516 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:39.304004 10516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:39.304148 10516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:39.305277 10516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "db64b9385a584c6abdb850bf38ed7bd8"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "1d3a9eb5e6052e73a1a17504e05fc383"
server_key_iv: "8e83d43f2fba832c9eebca0b2afc5f81"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.305665 10516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "db64b9385a584c6abdb850bf38ed7bd8"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "1d3a9eb5e6052e73a1a17504e05fc383"
server_key_iv: "8e83d43f2fba832c9eebca0b2afc5f81"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.307073 10516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:39.307904 10530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:39.308104 10516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:39.308210 10516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "db64b9385a584c6abdb850bf38ed7bd8"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "1d3a9eb5e6052e73a1a17504e05fc383"
server_key_iv: "8e83d43f2fba832c9eebca0b2afc5f81"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.308301 10516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:39.317217 10516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:39.317570 10516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:39.317739 10516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:39.321950 10516 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:39159
I20260928 17:56:39.321981 10582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:39159 every 8 connection(s)
I20260928 17:56:39.322353 10516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:39.323382 10583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:39.325872 10583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8: Bootstrap starting.
I20260928 17:56:39.326447 10583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:39.326690 10583 log.cc:824] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:39.327510 10583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8: No bootstrap required, opened a new log
I20260928 17:56:39.329272 10583 raft_consensus.cc:348] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } }
I20260928 17:56:39.329432 10583 raft_consensus.cc:374] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:39.329454 10583 raft_consensus.cc:729] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db64b9385a584c6abdb850bf38ed7bd8, State: Initialized, Role: FOLLOWER
I20260928 17:56:39.329607 10583 consensus_queue.cc:260] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } }
I20260928 17:56:39.329690 10583 raft_consensus.cc:388] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:39.329718 10583 raft_consensus.cc:482] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:39.329748 10583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:39.330317 10583 raft_consensus.cc:504] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } }
I20260928 17:56:39.330405 10583 leader_election.cc:302] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db64b9385a584c6abdb850bf38ed7bd8; no voters:
I20260928 17:56:39.330559 10583 leader_election.cc:288] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:39.330730 10586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:39.330827 10583 sys_catalog.cc:564] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:39.330921 10586 raft_consensus.cc:686] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [term 1 LEADER]: Becoming Leader. State: Replica: db64b9385a584c6abdb850bf38ed7bd8, State: Running, Role: LEADER
I20260928 17:56:39.331038 10586 consensus_queue.cc:237] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } }
I20260928 17:56:39.331478 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10516
I20260928 17:56:39.331581 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:39.331769 5305 external_mini_cluster.cc:1428] Setting key 3710b49fcc2f04598b8b5f2eca75e9a9
I20260928 17:56:39.332396 10587 sys_catalog.cc:455] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db64b9385a584c6abdb850bf38ed7bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } } }
I20260928 17:56:39.332502 10587 sys_catalog.cc:458] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.332719 10601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:39.332412 10588 sys_catalog.cc:455] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db64b9385a584c6abdb850bf38ed7bd8. Latest consensus state: current_term: 1 leader_uuid: "db64b9385a584c6abdb850bf38ed7bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db64b9385a584c6abdb850bf38ed7bd8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39159 } } }
I20260928 17:56:39.333168 10588 sys_catalog.cc:458] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.333246 10601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:39.334931 10601 catalog_manager.cc:1347] Generated new cluster ID: eca342ed52674501bf2a70917a689463
I20260928 17:56:39.334983 10601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:39.338490 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.332429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.5.46.126:39159)
0928 17:56:39.332813 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.332824 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.332859 (+ 35us) server_negotiation.cc:407] Connection header received
0928 17:56:39.332989 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.333008 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.333146 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.333420 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:39.333651 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.334907 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.336139 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.336746 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.336863 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.337617 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:39.337621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:39.337621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:39.337858 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:39.337898 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:39.337912 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:39.337976 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:39.338075 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 17:56:39.338267 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":76,"threads_started":1}
I20260928 17:56:39.339412 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:56:39.344156 10601 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:39.344676 10601 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:39.350450 10601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db64b9385a584c6abdb850bf38ed7bd8: Generated new TSK 0
I20260928 17:56:39.350659 10601 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 17:56:39.370442 10613 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 17:56:39.370543 10613 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 17:56:39.370887 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.355594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.5.46.126:39159)
0928 17:56:39.355646 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.355648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.355792 (+ 144us) server_negotiation.cc:407] Connection header received
0928 17:56:39.355912 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.355917 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.355929 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.355996 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0928 17:56:39.356163 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.357018 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.358276 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.358503 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.358569 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.358766 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 17:56:39.370691 (+ 11925us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 17:56:39.370833 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 17:56:39.370959 10600 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 17:56:39.371078 10612 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:39.355510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:39159 (local address 127.0.0.1:46544)
0928 17:56:39.355697 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:39.355718 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:39.355791 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:39.356025 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:39.356026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:39.356028 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 17:56:39.356092 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.356095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.357163 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.357164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.358148 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.358153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.358595 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.358596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.358659 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.358665 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 17:56:39.370825 (+ 12160us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 17:56:39.370970 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.46.126:39159: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":98,"threads_started":1}
I20260928 17:56:39.371579 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10516
2026-09-28T17:56:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (212 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 17:56:39.409502 5305 mini_oidc.cc:152] Starting JWKS server
I20260928 17:56:39.410239 5305 webserver.cc:460] Webserver started at https://127.0.0.1:33901/ using document root <none> and password file <none>
I20260928 17:56:39.410326 5305 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 17:56:39.410506 5305 webserver.cc:460] Webserver started at http://127.0.0.1:42467/ using document root <none> and password file <none>
2026-09-28T17:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:39Z Disabled control of system clock
I20260928 17:56:39.418475 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39971
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:39971
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33901/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 17:56:39.500409 10619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:39.500644 10619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:39.500667 10619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:39.501947 10619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:39.501994 10619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:39.502007 10619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:39.502020 10619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:39.502032 10619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 17:56:39.502045 10619 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 17:56:39.502097 10619 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33901/jwks.json
I20260928 17:56:39.503549 10619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:39971
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33901/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:39.503822 10619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:39.504184 10619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:39.509119 10624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.509151 10625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.509155 10627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:39.509349 10619 server_base.cc:1034] running on GCE node
I20260928 17:56:39.509732 10619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:39.510063 10619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:39.511219 10619 hybrid_clock.cc:648] HybridClock initialized: now 1790618199511195 us; error 37 us; skew 500 ppm
I20260928 17:56:39.512682 10619 webserver.cc:460] Webserver started at http://127.5.46.126:37679/ using document root <none> and password file <none>
I20260928 17:56:39.512954 10619 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:39.513013 10619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:39.513182 10619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:39.514361 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "f12f831c493544fa84ad814baf90cb20"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "78a26714950d1502ee6fc2fe4c107d72"
server_key_iv: "e75e27e9855c6308b7e04eafef3c43b8"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.514757 10619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "f12f831c493544fa84ad814baf90cb20"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "78a26714950d1502ee6fc2fe4c107d72"
server_key_iv: "e75e27e9855c6308b7e04eafef3c43b8"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.516405 10619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 17:56:39.517262 10633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:39.517491 10619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:39.517587 10619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "f12f831c493544fa84ad814baf90cb20"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "78a26714950d1502ee6fc2fe4c107d72"
server_key_iv: "e75e27e9855c6308b7e04eafef3c43b8"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.517669 10619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:39.528147 10619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:39.528514 10619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:39.528672 10619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:39.533053 10619 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:39971
I20260928 17:56:39.533087 10685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:39971 every 8 connection(s)
I20260928 17:56:39.533474 10619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:39.534195 10686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:39.537102 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20: Bootstrap starting.
I20260928 17:56:39.537716 10686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:39.538018 10686 log.cc:824] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:39.538818 10686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20: No bootstrap required, opened a new log
I20260928 17:56:39.540642 10686 raft_consensus.cc:348] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } }
I20260928 17:56:39.540830 10686 raft_consensus.cc:374] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:39.540882 10686 raft_consensus.cc:729] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12f831c493544fa84ad814baf90cb20, State: Initialized, Role: FOLLOWER
I20260928 17:56:39.541047 10686 consensus_queue.cc:260] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } }
I20260928 17:56:39.541131 10686 raft_consensus.cc:388] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:39.541177 10686 raft_consensus.cc:482] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:39.541235 10686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:39.541846 10686 raft_consensus.cc:504] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } }
I20260928 17:56:39.541989 10686 leader_election.cc:302] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f12f831c493544fa84ad814baf90cb20; no voters:
I20260928 17:56:39.542193 10686 leader_election.cc:288] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:39.542299 10689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:39.542461 10689 raft_consensus.cc:686] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [term 1 LEADER]: Becoming Leader. State: Replica: f12f831c493544fa84ad814baf90cb20, State: Running, Role: LEADER
I20260928 17:56:39.542490 10686 sys_catalog.cc:564] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:39.542577 10689 consensus_queue.cc:237] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } }
I20260928 17:56:39.543128 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10619
I20260928 17:56:39.543242 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:39.543226 10690 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f12f831c493544fa84ad814baf90cb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } } }
I20260928 17:56:39.543311 10690 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.543255 10691 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f12f831c493544fa84ad814baf90cb20. Latest consensus state: current_term: 1 leader_uuid: "f12f831c493544fa84ad814baf90cb20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f12f831c493544fa84ad814baf90cb20" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39971 } } }
I20260928 17:56:39.543394 10691 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.543486 5305 external_mini_cluster.cc:1428] Setting key 52884d3ebf273f28c445e8d4663a5758
I20260928 17:56:39.543536 10698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:39.544566 10698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:39.546237 10698 catalog_manager.cc:1347] Generated new cluster ID: 116c1d86a66447f8943d5ccf540843d1
I20260928 17:56:39.546305 10698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:39.549129 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.544061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.5.46.126:39971)
0928 17:56:39.544292 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.544305 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.544346 (+ 41us) server_negotiation.cc:407] Connection header received
0928 17:56:39.544488 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.544496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.544599 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.544773 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:39.544967 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.546084 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.546981 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.547469 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.547613 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.548364 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:39.548369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:39.548369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:39.548557 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:39.548596 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:39.548608 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:39.548686 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:39.548770 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 17:56:39.548954 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260928 17:56:39.550163 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:56:39.563396 10698 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:39.564050 10698 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:39.574326 10698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f12f831c493544fa84ad814baf90cb20: Generated new TSK 0
I20260928 17:56:39.574594 10698 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:56:39.585096 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.581069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.5.46.126:39971)
0928 17:56:39.581120 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.581122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.581231 (+ 109us) server_negotiation.cc:407] Connection header received
0928 17:56:39.581284 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.581285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.581298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.581392 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0928 17:56:39.581522 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.582310 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.583185 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.583508 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.583586 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.583875 (+ 289us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 17:56:39.584881 (+ 1006us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33901/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 17:56:39.585019 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33901/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260928 17:56:39.585168 10704 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60010: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33901/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260928 17:56:39.585472 10715 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:39.580981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:39971 (local address 127.0.0.1:60010)
0928 17:56:39.581134 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:39.581147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:39.581220 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:39.581391 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:39.581392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:39.581394 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 17:56:39.581446 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.581448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.582414 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.582415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.583084 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.583086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.583617 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.583619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.583684 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.583690 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 17:56:39.584998 (+ 1308us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33901/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 17:56:39.585105 (+ 107us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.46.126:39971: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33901/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":103,"threads_started":1}
I20260928 17:56:39.585704 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10619
E20260928 17:56:39.596267 10716 webserver.cc:533] Webserver: sslize failed:
E20260928 17:56:39.596374 10716 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-28T17:56:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T17:56:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:39Z Disabled control of system clock
I20260928 17:56:39.631966 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38561
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:42253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:38561
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/testchainca.cert with env {}
W20260928 17:56:39.713405 10720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:39.713662 10720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:39.713702 10720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:39.714895 10720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:39.714967 10720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:39.714987 10720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:39.715008 10720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:39.715039 10720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 17:56:39.715067 10720 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 17:56:39.715097 10720 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 17:56:39.716615 10720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:38561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:39.716917 10720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:39.717149 10720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:39.722198 10726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.722183 10728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:39.722182 10725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:39.722487 10720 server_base.cc:1034] running on GCE node
I20260928 17:56:39.722747 10720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:39.723048 10720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:39.724263 10720 hybrid_clock.cc:648] HybridClock initialized: now 1790618199724249 us; error 31 us; skew 500 ppm
I20260928 17:56:39.725613 10720 webserver.cc:460] Webserver started at http://127.5.46.126:42381/ using document root <none> and password file <none>
I20260928 17:56:39.725860 10720 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:39.725931 10720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:39.726089 10720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:39.727193 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "459a4404ae0c4085af6f292985a3fceb"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "a2723c036895c55fc4860f6e71359a0f"
server_key_iv: "e3fc30b99cf84e6307246eacedb06690"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.727579 10720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "459a4404ae0c4085af6f292985a3fceb"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "a2723c036895c55fc4860f6e71359a0f"
server_key_iv: "e3fc30b99cf84e6307246eacedb06690"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.729101 10720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:39.729956 10734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:39.730155 10720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:39.730315 10720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "459a4404ae0c4085af6f292985a3fceb"
format_stamp: "Formatted at 2026-09-28 17:56:39 on dist-test-slave-z2t3"
server_key: "a2723c036895c55fc4860f6e71359a0f"
server_key_iv: "e3fc30b99cf84e6307246eacedb06690"
server_key_version: "encryptionkey@0"
I20260928 17:56:39.730430 10720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:39.768484 10720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:39.768862 10720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:39.769032 10720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:39.773134 10720 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:38561
I20260928 17:56:39.773202 10786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:38561 every 8 connection(s)
I20260928 17:56:39.773603 10720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:39.774616 10787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:39.777174 10787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb: Bootstrap starting.
I20260928 17:56:39.777190 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10720
I20260928 17:56:39.777321 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:39.777532 5305 external_mini_cluster.cc:1428] Setting key 8858162942bfef75eeac25445b1fb025
I20260928 17:56:39.778038 10787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:39.778442 10787 log.cc:824] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:39.779352 10787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb: No bootstrap required, opened a new log
I20260928 17:56:39.781854 10787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } }
I20260928 17:56:39.782086 10787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:39.782117 10787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 459a4404ae0c4085af6f292985a3fceb, State: Initialized, Role: FOLLOWER
I20260928 17:56:39.782279 10787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } }
I20260928 17:56:39.782347 10787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:39.782371 10787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:39.782450 10787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:39.783329 10787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } }
I20260928 17:56:39.783463 10787 leader_election.cc:302] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 459a4404ae0c4085af6f292985a3fceb; no voters:
I20260928 17:56:39.783684 10787 leader_election.cc:288] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:39.783738 10792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:39.783906 10792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [term 1 LEADER]: Becoming Leader. State: Replica: 459a4404ae0c4085af6f292985a3fceb, State: Running, Role: LEADER
I20260928 17:56:39.783993 10792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } }
I20260928 17:56:39.784024 10787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:39.784329 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.778152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34758 (local address 127.5.46.126:38561)
0928 17:56:39.778430 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.778438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.778464 (+ 26us) server_negotiation.cc:407] Connection header received
0928 17:56:39.779101 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.779110 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.779181 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.779315 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:39.779605 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.780587 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.781900 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.782531 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.782639 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.783393 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:39.783397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:39.783397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:56:39.783649 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:39.783686 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:56:39.783698 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:39.783774 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:39.784040 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 17:56:39.784130 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":105,"threads_started":1}
I20260928 17:56:39.784964 10794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 459a4404ae0c4085af6f292985a3fceb. Latest consensus state: current_term: 1 leader_uuid: "459a4404ae0c4085af6f292985a3fceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } } }
I20260928 17:56:39.785053 10794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.785267 10802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:39.785460 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:56:39.785753 10802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:39.786226 10793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "459a4404ae0c4085af6f292985a3fceb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "459a4404ae0c4085af6f292985a3fceb" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38561 } } }
I20260928 17:56:39.786326 10793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:39.787515 10802 catalog_manager.cc:1347] Generated new cluster ID: a88157ab97274d6da91c48d9db1c3920
I20260928 17:56:39.787575 10802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:39.802681 10802 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:39.803283 10802 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:39.812748 10802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 459a4404ae0c4085af6f292985a3fceb: Generated new TSK 0
I20260928 17:56:39.813005 10802 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:56:39.919390 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:39.819560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.5.46.126:38561)
0928 17:56:39.819622 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:39.819624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:39.819729 (+ 105us) server_negotiation.cc:407] Connection header received
0928 17:56:39.819782 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:39.819783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:39.819796 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:39.819920 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0928 17:56:39.820043 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.820844 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.821635 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.821988 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.822060 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.822280 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 17:56:39.919112 (+ 96832us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 17:56:39.919297 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34762: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":36}
W20260928 17:56:39.919456 10790 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34762: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 17:56:39.919680 10816 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:39.819475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38561 (local address 127.0.0.1:34762)
0928 17:56:39.819644 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:39.819655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:39.819725 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:39.819926 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:39.819927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:39.819929 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 17:56:39.819981 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.819983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.820934 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.820935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.821553 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:39.821556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:39.822098 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:39.822099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:39.822157 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:39.822164 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 17:56:39.919277 (+ 97113us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 17:56:39.919483 (+ 206us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.46.126:38561: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260928 17:56:39.920094 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10720
2026-09-28T17:56:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (314 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:40 dist-test-slave-z2t3 krb5kdc[10828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:40 dist-test-slave-z2t3 krb5kdc[10828](info): set up 2 sockets
Sep 28 17:56:40 dist-test-slave-z2t3 krb5kdc[10828](info): commencing operation
krb5kdc: starting...
W20260928 17:56:41.152185 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:41 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:41.279070 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35505
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:35505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:41.360697 10841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:41.360908 10841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:41.360929 10841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:41.362067 10841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:41.362103 10841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:41.362118 10841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:41.362129 10841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:41.362141 10841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:41.363538 10841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:35505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:41.363741 10841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:41.363942 10841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:41.369010 10841 server_base.cc:1034] running on GCE node
W20260928 17:56:41.368906 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:41.368906 10846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:41.368973 10847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:41.369482 10841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:41.369805 10841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:41.370997 10841 hybrid_clock.cc:648] HybridClock initialized: now 1790618201370989 us; error 53 us; skew 500 ppm
Sep 28 17:56:41 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618201, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:41.380151 10841 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:41.380739 10841 webserver.cc:460] Webserver started at http://127.5.46.126:40977/ using document root <none> and password file <none>
I20260928 17:56:41.381004 10841 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:41.381081 10841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:41.381235 10841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:41.382380 10841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "1a5569fe33bb4bdd935731ee9788fef3"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "084e3b7ff77087baa5eb69557fd3bce8"
server_key_iv: "28a85d7918eeb4a6cd2ac99e2c28b787"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.382766 10841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "1a5569fe33bb4bdd935731ee9788fef3"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "084e3b7ff77087baa5eb69557fd3bce8"
server_key_iv: "28a85d7918eeb4a6cd2ac99e2c28b787"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.384194 10841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:41.385030 10856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:41.385217 10841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:41.385363 10841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "1a5569fe33bb4bdd935731ee9788fef3"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "084e3b7ff77087baa5eb69557fd3bce8"
server_key_iv: "28a85d7918eeb4a6cd2ac99e2c28b787"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.385447 10841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:41.393687 10841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:41.395921 10841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:41.396087 10841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:41.400197 10841 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:35505
I20260928 17:56:41.400251 10908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:35505 every 8 connection(s)
I20260928 17:56:41.400625 10841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:41.401546 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:41.403801 10909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: Bootstrap starting.
I20260928 17:56:41.404171 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10841
I20260928 17:56:41.404284 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:41.404482 5305 external_mini_cluster.cc:1428] Setting key 22641155dd5aad908fc1437f55f996c2
I20260928 17:56:41.404518 10909 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:41.404855 10909 log.cc:824] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:41.405616 10909 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: No bootstrap required, opened a new log
I20260928 17:56:41.408063 10909 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } }
I20260928 17:56:41.408244 10909 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:41.408291 10909 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a5569fe33bb4bdd935731ee9788fef3, State: Initialized, Role: FOLLOWER
I20260928 17:56:41.408427 10909 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } }
I20260928 17:56:41.408505 10909 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:41.408528 10909 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:41.408594 10909 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:41.409211 10909 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } }
I20260928 17:56:41.409315 10909 leader_election.cc:302] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a5569fe33bb4bdd935731ee9788fef3; no voters:
I20260928 17:56:41.409462 10909 leader_election.cc:288] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:41.409524 10914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:41.409718 10914 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [term 1 LEADER]: Becoming Leader. State: Replica: 1a5569fe33bb4bdd935731ee9788fef3, State: Running, Role: LEADER
I20260928 17:56:41.409715 10909 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:41.409852 10914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } }
I20260928 17:56:41.410285 10915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a5569fe33bb4bdd935731ee9788fef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } } }
I20260928 17:56:41.410429 10915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:41.410526 10916 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a5569fe33bb4bdd935731ee9788fef3. Latest consensus state: current_term: 1 leader_uuid: "1a5569fe33bb4bdd935731ee9788fef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a5569fe33bb4bdd935731ee9788fef3" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35505 } } }
I20260928 17:56:41.411825 10916 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3 [sys.catalog]: This master's current role is: LEADER
W20260928 17:56:41.411579 10928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:56:41.412161 10928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 28 17:56:41 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:41.416204 10931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:41.416366 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:41.404994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39152 (local address 127.5.46.126:35505)
0928 17:56:41.405166 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:41.405176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:41.405237 (+ 61us) server_negotiation.cc:407] Connection header received
0928 17:56:41.405823 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:41.405832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:41.405896 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:41.406057 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:41.406647 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.408146 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.409096 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.409732 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.409851 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:41.412976 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:41.412980 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:41.412980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:41.412993 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:41.414718 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:41.415105 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:41.415109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:41.415109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:41.415202 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:41.415417 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:41.415418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:41.415418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:41.415662 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:41.415885 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:41.415967 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 17:56:41.416110 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260928 17:56:41.416461 10931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:41.418123 10931 catalog_manager.cc:1347] Generated new cluster ID: ed67765082c84930b8cab9173718f6ec
I20260928 17:56:41.418220 10931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:41.436342 10931 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:41.436971 10931 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:41.443074 10931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a5569fe33bb4bdd935731ee9788fef3: Generated new TSK 0
I20260928 17:56:41.443336 10931 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:41.484455 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:41.569289 10935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:41.569552 10935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:41.569593 10935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:41.570828 10935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:41.570911 10935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:41.570955 10935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:41.572582 10935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:41.572885 10935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:41.573134 10935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:41.578564 10941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:41.578708 10943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:41.578564 10940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:41.578828 10935 server_base.cc:1034] running on GCE node
I20260928 17:56:41.579036 10935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:41.579321 10935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:41.580513 10935 hybrid_clock.cc:648] HybridClock initialized: now 1790618201580485 us; error 45 us; skew 500 ppm
Sep 28 17:56:41 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618201, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:41.583747 10935 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:41.584405 10935 webserver.cc:460] Webserver started at http://127.5.46.65:39197/ using document root <none> and password file <none>
I20260928 17:56:41.584714 10935 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:41.584769 10935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:41.584971 10935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:41.586140 10935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "96cd9a0db467483a97901bc3d2f1891d"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "55b65d58b06bd503a122a76cf51c67c7"
server_key_iv: "8d39d91631ec739b39cda55acf9095a1"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.586534 10935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "96cd9a0db467483a97901bc3d2f1891d"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "55b65d58b06bd503a122a76cf51c67c7"
server_key_iv: "8d39d91631ec739b39cda55acf9095a1"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.588179 10935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:41.589275 10950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:41.589630 10935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:41.589733 10935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "96cd9a0db467483a97901bc3d2f1891d"
format_stamp: "Formatted at 2026-09-28 17:56:41 on dist-test-slave-z2t3"
server_key: "55b65d58b06bd503a122a76cf51c67c7"
server_key_iv: "8d39d91631ec739b39cda55acf9095a1"
server_key_version: "encryptionkey@0"
I20260928 17:56:41.589860 10935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:41.604943 10935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:41.608022 10935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:41.608198 10935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:41.608500 10935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:41.608902 10935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:41.608942 10935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:41.609005 10935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:41.609033 10935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:41.616242 10935 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33739
I20260928 17:56:41.616286 11063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33739 every 8 connection(s)
I20260928 17:56:41.616674 10935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:41.620213 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 10935
I20260928 17:56:41.620317 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:41.620500 5305 external_mini_cluster.cc:1428] Setting key 7f9c77729a41ff298b088d46df364ded
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Sep 28 17:56:41 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618201, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:41.632993 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:41.617719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:54693 (local address 127.5.46.126:35505)
0928 17:56:41.617846 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:41.617848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:41.618041 (+ 193us) server_negotiation.cc:407] Connection header received
0928 17:56:41.619084 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:41.619086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:41.619099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:41.619159 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:41.619871 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.620647 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.622789 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.623132 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.623218 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:41.629822 (+ 6604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:41.629824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:41.629825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:41.629834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:41.631405 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:41.631968 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:41.631969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:41.631969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:41.632032 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:41.632298 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:41.632299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:41.632299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:41.632431 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:41.632541 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:41.632837 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 17:56:41.632915 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260928 17:56:41.633267 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:41.617672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35505 (local address 127.5.46.65:54693)
0928 17:56:41.617889 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:41.617939 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:41.618955 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:41.619246 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:41.619250 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:41.619523 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:41.619721 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:41.619725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.620764 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.620766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:41.622675 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:41.622679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:41.623245 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:41.623246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:41.623429 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:41.628078 (+ 4649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:41.628089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:41.629674 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:41.631535 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:41.631544 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:41.631546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:41.631811 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:41.632100 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:41.632101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:41.632101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:41.632226 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:41.632537 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:41.632545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:41.632691 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 17:56:41.632795 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 17:56:41.632961 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":91,"threads_started":1}
I20260928 17:56:41.633944 11064 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35505
I20260928 17:56:41.634042 11064 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:41.634208 11064 heartbeater.cc:511] Master 127.5.46.126:35505 requested a full tablet report, sending...
I20260928 17:56:41.634894 10873 ts_manager.cc:194] Registered new tserver with Master: 96cd9a0db467483a97901bc3d2f1891d (127.5.46.65:33739)
I20260928 17:56:41.635637 10873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:54693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
I20260928 17:56:42.636987 11064 heartbeater.cc:503] Master 127.5.46.126:35505 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
W20260928 17:56:42.801010 5305 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.46.66: real 1.180s user 0.001s sys 0.002s
I20260928 17:56:42.801431 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:42.885129 11069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:42.885366 11069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:42.885392 11069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:42.886652 11069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:42.886703 11069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:42.886729 11069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:42.888379 11069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:42.888651 11069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:42.888948 11069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:42.895723 11075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:42.895723 11077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:42.895996 11074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:42.896003 11069 server_base.cc:1034] running on GCE node
I20260928 17:56:42.896308 11069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:42.896611 11069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:42.897821 11069 hybrid_clock.cc:648] HybridClock initialized: now 1790618202897803 us; error 43 us; skew 500 ppm
Sep 28 17:56:42 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:42.900851 11069 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:42.901552 11069 webserver.cc:460] Webserver started at http://127.5.46.66:43751/ using document root <none> and password file <none>
I20260928 17:56:42.901820 11069 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:42.901870 11069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:42.902057 11069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:42.903249 11069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "3de0a463adda40db8ffb006ba110d958"
format_stamp: "Formatted at 2026-09-28 17:56:42 on dist-test-slave-z2t3"
server_key: "4d05c4342ea38e3996659dd19a4f63cd"
server_key_iv: "1f2d7baf3fab6388514c86d0b82a1f0a"
server_key_version: "encryptionkey@0"
I20260928 17:56:42.903616 11069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "3de0a463adda40db8ffb006ba110d958"
format_stamp: "Formatted at 2026-09-28 17:56:42 on dist-test-slave-z2t3"
server_key: "4d05c4342ea38e3996659dd19a4f63cd"
server_key_iv: "1f2d7baf3fab6388514c86d0b82a1f0a"
server_key_version: "encryptionkey@0"
I20260928 17:56:42.905110 11069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:42.906049 11084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:42.906338 11069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:42.906450 11069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "3de0a463adda40db8ffb006ba110d958"
format_stamp: "Formatted at 2026-09-28 17:56:42 on dist-test-slave-z2t3"
server_key: "4d05c4342ea38e3996659dd19a4f63cd"
server_key_iv: "1f2d7baf3fab6388514c86d0b82a1f0a"
server_key_version: "encryptionkey@0"
I20260928 17:56:42.906546 11069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:42.916708 11069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:42.918987 11069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:42.919175 11069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:42.919488 11069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:42.919886 11069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:42.919960 11069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:42.920001 11069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:42.920032 11069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:42.926384 11069 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:39973
I20260928 17:56:42.926425 11197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:39973 every 8 connection(s)
I20260928 17:56:42.926807 11069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:42 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:42.936687 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11069
I20260928 17:56:42.936827 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:42.937038 5305 external_mini_cluster.cc:1428] Setting key 672fee1e0489a413bc4fb7fbb06549e7
I20260928 17:56:42.937357 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:42.927606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35505 (local address 127.5.46.66:60571)
0928 17:56:42.927778 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:42.927823 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:42.928763 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:42.929018 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:42.929021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:42.929341 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:42.929538 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:42.929543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:42.930288 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:42.930288 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:42.931148 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:42.931150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:42.931557 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:42.931558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:42.931670 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:42.932295 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:42.932304 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:42.933768 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:42.935569 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:42.935576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:42.935578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:42.935832 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:42.936189 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:42.936190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:42.936190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:42.936303 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:42.936591 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:42.936599 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:42.936796 (+ 197us) client_negotiation.cc:769] Sending connection context
0928 17:56:42.936927 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 17:56:42.937084 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260928 17:56:42.937469 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:42.927615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:60571 (local address 127.5.46.126:35505)
0928 17:56:42.927736 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:42.927738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:42.927936 (+ 198us) server_negotiation.cc:407] Connection header received
0928 17:56:42.928870 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:42.928871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:42.928885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:42.928960 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:42.929632 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:42.930143 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:42.931255 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:42.931467 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:42.931534 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:42.933898 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:42.933900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:42.933900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:42.933908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:42.935487 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:42.936051 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:42.936057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:42.936057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:42.936123 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:42.936375 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:42.936376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:42.936376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:42.936502 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:42.936588 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:42.937197 (+ 609us) server_negotiation.cc:299] Negotiation successful
0928 17:56:42.937294 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260928 17:56:42.937817 11198 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35505
I20260928 17:56:42.937942 11198 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:42.938153 11198 heartbeater.cc:511] Master 127.5.46.126:35505 requested a full tablet report, sending...
I20260928 17:56:42.938570 10873 ts_manager.cc:194] Registered new tserver with Master: 3de0a463adda40db8ffb006ba110d958 (127.5.46.66:39973)
I20260928 17:56:42.938972 10873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:60571
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
W20260928 17:56:43.431414 5305 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.5.46.67: real 0.493s user 0.000s sys 0.003s
I20260928 17:56:43.431815 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:43.514410 11204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:43.514673 11204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:43.514715 11204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:43.515969 11204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:43.516042 11204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:43.516088 11204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:43.517665 11204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:43.517951 11204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:43.518201 11204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:43.523348 11210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:43.523348 11212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:43.523348 11209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:43.523691 11204 server_base.cc:1034] running on GCE node
I20260928 17:56:43.523973 11204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:43.524231 11204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:43.525420 11204 hybrid_clock.cc:648] HybridClock initialized: now 1790618203525406 us; error 39 us; skew 500 ppm
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.528013 11204 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:43.528663 11204 webserver.cc:460] Webserver started at http://127.5.46.67:40515/ using document root <none> and password file <none>
I20260928 17:56:43.528929 11204 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:43.528995 11204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:43.529131 11204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:43.530184 11204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "1a795a7f6e5c4a66bc74483f6a56a77a"
format_stamp: "Formatted at 2026-09-28 17:56:43 on dist-test-slave-z2t3"
server_key: "6079df8455688be8790b129da1abc6d4"
server_key_iv: "0392bf8f2b1ae86b2ec6294b92883c13"
server_key_version: "encryptionkey@0"
I20260928 17:56:43.530558 11204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "1a795a7f6e5c4a66bc74483f6a56a77a"
format_stamp: "Formatted at 2026-09-28 17:56:43 on dist-test-slave-z2t3"
server_key: "6079df8455688be8790b129da1abc6d4"
server_key_iv: "0392bf8f2b1ae86b2ec6294b92883c13"
server_key_version: "encryptionkey@0"
I20260928 17:56:43.531975 11204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:43.532823 11219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:43.533002 11204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:43.533107 11204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "1a795a7f6e5c4a66bc74483f6a56a77a"
format_stamp: "Formatted at 2026-09-28 17:56:43 on dist-test-slave-z2t3"
server_key: "6079df8455688be8790b129da1abc6d4"
server_key_iv: "0392bf8f2b1ae86b2ec6294b92883c13"
server_key_version: "encryptionkey@0"
I20260928 17:56:43.533201 11204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:43.557248 11204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:43.559587 11204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:43.559780 11204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:43.560068 11204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:43.560390 11204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:43.560442 11204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:43.560482 11204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:43.560523 11204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:43.566553 11204 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45695
I20260928 17:56:43.566644 11332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45695 every 8 connection(s)
I20260928 17:56:43.567003 11204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:43.567318 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11204
I20260928 17:56:43.567430 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:43.567642 5305 external_mini_cluster.cc:1428] Setting key 4a53f5ae7f42a1c2532138b78b81ecfe
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:43.577639 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:43.567995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:46609 (local address 127.5.46.126:35505)
0928 17:56:43.568234 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.568236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.568256 (+ 20us) server_negotiation.cc:407] Connection header received
0928 17:56:43.569396 (+ 1140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.569397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.569409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.569470 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.570091 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.570683 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.571837 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.572124 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.572192 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:43.574687 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:43.574689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:43.574690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:43.574697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:43.576215 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.576644 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.576645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.576645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.576704 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.577012 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.577012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.577013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.577127 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:43.577180 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:43.577491 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 17:56:43.577564 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":66,"threads_started":1}
I20260928 17:56:43.577950 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:43.567907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35505 (local address 127.5.46.67:46609)
0928 17:56:43.568153 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:43.568192 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:43.569290 (+ 1098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:43.569529 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:43.569532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:43.569802 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:43.570006 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:43.570010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.570804 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.570805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:43.571735 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:43.571738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.572212 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.572213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:43.572392 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:43.573032 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:43.573042 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:43.574566 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:43.576301 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:43.576308 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:43.576310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:43.576561 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:43.576810 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:43.576811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:43.576811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:43.576948 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:43.577188 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:43.577194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:43.577333 (+ 139us) client_negotiation.cc:769] Sending connection context
0928 17:56:43.577453 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 17:56:43.577672 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260928 17:56:43.578540 11333 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35505
I20260928 17:56:43.578655 11333 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:43.578840 11333 heartbeater.cc:511] Master 127.5.46.126:35505 requested a full tablet report, sending...
I20260928 17:56:43.579239 10873 ts_manager.cc:194] Registered new tserver with Master: 1a795a7f6e5c4a66bc74483f6a56a77a (127.5.46.67:45695)
I20260928 17:56:43.579617 10873 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:46609
I20260928 17:56:43.579643 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/test-admin.keytab.
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.596608 5305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 17:56:43.599368 11341 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:43.598658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.5.46.65:33739)
0928 17:56:43.598784 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.598791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.598806 (+ 15us) server_negotiation.cc:407] Connection header received
0928 17:56:43.598836 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.598838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.598859 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.598935 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.599284 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33930: BlockingRecv error: recv got EOF from 127.0.0.1:33930 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
W20260928 17:56:43.599476 5305 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc)
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.600524 5305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.601383 5305 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:43.613821 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:43.601954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.5.46.65:33739)
0928 17:56:43.602014 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.602016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.602050 (+ 34us) server_negotiation.cc:407] Connection header received
0928 17:56:43.602129 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.602130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.602142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.602192 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.602592 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.603298 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.604106 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.604448 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.604527 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:43.606748 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:43.606758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:43.606758 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:43.606770 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:43.612236 (+ 5466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.612790 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.612796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.612796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.612870 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.613101 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.613102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.613102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.613295 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:43.613478 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:43.613621 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 17:56:43.613706 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260928 17:56:43.616046 11341 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:43.615462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.5.46.65:33739)
0928 17:56:43.615518 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.615520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.615528 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:43.615547 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.615548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.615558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.615610 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.615995 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33950: BlockingRecv error: recv got EOF from 127.0.0.1:33950 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260928 17:56:43.616102 5305 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc' of type FILE
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.617280 5305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.618340 5305 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:43.627460 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:43.618965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.5.46.65:33739)
0928 17:56:43.619014 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.619031 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.619155 (+ 124us) server_negotiation.cc:407] Connection header received
0928 17:56:43.619180 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.619181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.619191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.619250 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.619762 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.620473 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.621196 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.621589 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.621660 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:43.624604 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:43.624605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:43.624606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:43.624615 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:43.626235 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.626634 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.626635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.626636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.626695 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.626885 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.626886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.626886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.627020 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:43.627113 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:43.627257 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 17:56:43.627342 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 17:56:43.629146 11341 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:43.628573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33968 (local address 127.5.46.65:33739)
0928 17:56:43.628613 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.628615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.628667 (+ 52us) server_negotiation.cc:407] Connection header received
0928 17:56:43.628741 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.628741 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.628751 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.628807 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.629091 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33968: BlockingRecv error: recv got EOF from 127.0.0.1:33968 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 17:56:43.629230 5305 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790618177091767-5305-0/krb5kdc/krb5cc' of type FILE
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.630417 5305 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:43.631556 5305 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[10828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.65@KRBTEST.COM
I20260928 17:56:43.639775 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:43.632162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.5.46.65:33739)
0928 17:56:43.632215 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:43.632217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:43.632251 (+ 34us) server_negotiation.cc:407] Connection header received
0928 17:56:43.632308 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:43.632308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:43.632316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:43.632434 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:43.632801 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.633260 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.633945 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:43.634213 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:43.634289 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:43.636823 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:43.636825 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:43.636825 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:43.636833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:43.638447 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.638916 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.638917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.638918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.638980 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:43.639202 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:43.639203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:43.639203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:43.639310 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:43.639415 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:43.639568 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 17:56:43.639664 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 17:56:43.640532 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10935
I20260928 17:56:43.647352 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11069
I20260928 17:56:43.653169 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11204
I20260928 17:56:43.657543 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 10841
2026-09-28T17:56:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3556 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[11354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[11354](info): set up 2 sockets
Sep 28 17:56:43 dist-test-slave-z2t3 krb5kdc[11354](info): commencing operation
krb5kdc: starting...
W20260928 17:56:44.696210 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:44 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab.
Entry for principal oryx/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab.
I20260928 17:56:44.819533 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:43449
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab
--principal=oryx/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:43449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:44.902030 11367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:44.902273 11367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:44.902298 11367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:44.903545 11367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:44.903604 11367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:44.903618 11367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:44.903631 11367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:44.903643 11367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:44.905246 11367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:43449
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab
--principal=oryx/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:43449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:44.905521 11367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:44.905782 11367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:44.911182 11375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:44.911227 11372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:44.911392 11367 server_base.cc:1034] running on GCE node
W20260928 17:56:44.911201 11373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:44.911644 11367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:44.911998 11367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:44.913154 11367 hybrid_clock.cc:648] HybridClock initialized: now 1790618204913149 us; error 38 us; skew 500 ppm
Sep 28 17:56:44 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:44.915889 11367 init.cc:377] Logged in from keytab as oryx/127.5.46.126@KRBTEST.COM (short username oryx)
I20260928 17:56:44.916358 11367 webserver.cc:460] Webserver started at http://127.5.46.126:45809/ using document root <none> and password file <none>
I20260928 17:56:44.916591 11367 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:44.916656 11367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:44.916805 11367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:44.917877 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "e2f96d049b4e415cb617f0844b970f30"
format_stamp: "Formatted at 2026-09-28 17:56:44 on dist-test-slave-z2t3"
server_key: "3864d450312e275a0db694fc26ff8257"
server_key_iv: "81bfa3115d6bccf74dfa2978f72077db"
server_key_version: "encryptionkey@0"
I20260928 17:56:44.918268 11367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "e2f96d049b4e415cb617f0844b970f30"
format_stamp: "Formatted at 2026-09-28 17:56:44 on dist-test-slave-z2t3"
server_key: "3864d450312e275a0db694fc26ff8257"
server_key_iv: "81bfa3115d6bccf74dfa2978f72077db"
server_key_version: "encryptionkey@0"
I20260928 17:56:44.919595 11367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 17:56:44.920393 11382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:44.920562 11367 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:44.920657 11367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e2f96d049b4e415cb617f0844b970f30"
format_stamp: "Formatted at 2026-09-28 17:56:44 on dist-test-slave-z2t3"
server_key: "3864d450312e275a0db694fc26ff8257"
server_key_iv: "81bfa3115d6bccf74dfa2978f72077db"
server_key_version: "encryptionkey@0"
I20260928 17:56:44.920733 11367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:44.950587 11367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:44.953186 11367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:44.953358 11367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:44.957496 11367 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:43449
I20260928 17:56:44.957547 11434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:43449 every 8 connection(s)
I20260928 17:56:44.957935 11367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:44.959007 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:44.961288 11435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Bootstrap starting.
I20260928 17:56:44.961823 11435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:44.962065 11435 log.cc:824] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:44.962704 11435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: No bootstrap required, opened a new log
I20260928 17:56:44.964396 11435 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } }
I20260928 17:56:44.964625 11435 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:44.964653 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11367
I20260928 17:56:44.964668 11435 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2f96d049b4e415cb617f0844b970f30, State: Initialized, Role: FOLLOWER
I20260928 17:56:44.964771 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:44.964831 11435 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } }
I20260928 17:56:44.964919 11435 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:44.964951 11435 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:44.964983 5305 external_mini_cluster.cc:1428] Setting key 124efe7a1b040d70279cbed60cd5a87d
I20260928 17:56:44.965001 11435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:44.966130 11435 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } }
I20260928 17:56:44.966264 11435 leader_election.cc:302] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2f96d049b4e415cb617f0844b970f30; no voters:
I20260928 17:56:44.966431 11435 leader_election.cc:288] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:44.966483 11440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:44.966626 11440 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 LEADER]: Becoming Leader. State: Replica: e2f96d049b4e415cb617f0844b970f30, State: Running, Role: LEADER
I20260928 17:56:44.966709 11435 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:44.966713 11440 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } }
I20260928 17:56:44.967064 11441 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } }
I20260928 17:56:44.967062 11442 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2f96d049b4e415cb617f0844b970f30. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } }
I20260928 17:56:44.967283 11442 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:44.967463 11441 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:44.967756 11449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:44.968314 11449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:44.970515 11449 catalog_manager.cc:1347] Generated new cluster ID: b2670d8e59524689a71ea3c6cb677f48
I20260928 17:56:44.970572 11449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:44 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:44.975795 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:44.965554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.5.46.126:43449)
0928 17:56:44.965741 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:44.965749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:44.965779 (+ 30us) server_negotiation.cc:407] Connection header received
0928 17:56:44.965911 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:44.965917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:44.965961 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:44.966070 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:44.966871 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:44.968473 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:44.969387 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:44.969990 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:44.970101 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:44.972569 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:44.972573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:44.972573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:44.972585 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:44.974271 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:44.974655 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:44.974663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:44.974663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:44.974763 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:44.974966 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:44.974967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:44.974968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:44.975224 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:44.975398 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:44.975472 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 17:56:44.975620 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
I20260928 17:56:44.992770 11449 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:44.993552 11449 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:45.003495 11449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Generated new TSK 0
I20260928 17:56:45.003759 11449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
I20260928 17:56:45.035967 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:45.122816 11461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:45.123113 11461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:45.123148 11461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:45.124905 11461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:45.124966 11461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:45.125006 11461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:45.127180 11461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:45.127457 11461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:45.127708 11461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:45.133231 11467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.133224 11461 server_base.cc:1034] running on GCE node
W20260928 17:56:45.133174 11466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.133175 11469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.133671 11461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.133953 11461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.135120 11461 hybrid_clock.cc:648] HybridClock initialized: now 1790618205135113 us; error 34 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.137789 11461 init.cc:377] Logged in from keytab as oryx/127.5.46.65@KRBTEST.COM (short username oryx)
I20260928 17:56:45.138258 11461 webserver.cc:460] Webserver started at http://127.5.46.65:38897/ using document root <none> and password file <none>
I20260928 17:56:45.138506 11461 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.138578 11461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.138720 11461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:45.139789 11461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "0e88892e17db468c992a1ce180ed66be"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "69c4ca1bb272157ebfd33db48d486adb"
server_key_iv: "13d8fffe2ed9c932ed2132d700fd317f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.140220 11461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "0e88892e17db468c992a1ce180ed66be"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "69c4ca1bb272157ebfd33db48d486adb"
server_key_iv: "13d8fffe2ed9c932ed2132d700fd317f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.141593 11461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:45.142509 11476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.142700 11461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:45.142797 11461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "0e88892e17db468c992a1ce180ed66be"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "69c4ca1bb272157ebfd33db48d486adb"
server_key_iv: "13d8fffe2ed9c932ed2132d700fd317f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.142892 11461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.172535 11461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.174947 11461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:45.175122 11461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.175375 11461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:45.175737 11461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:45.175798 11461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.175834 11461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:45.175905 11461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.181761 11461 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36481
I20260928 17:56:45.181805 11589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36481 every 8 connection(s)
I20260928 17:56:45.182178 11461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:45.191699 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11461
I20260928 17:56:45.191779 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.182933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:60483 (local address 127.5.46.126:43449)
0928 17:56:45.182982 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.182983 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.183215 (+ 232us) server_negotiation.cc:407] Connection header received
0928 17:56:45.183734 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.183735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.183752 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.183808 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.184822 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.185328 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.186337 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.186552 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.186632 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.188876 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.188877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.188878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.188886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.190380 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.190796 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.190798 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.190798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.190857 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.191102 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.191103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.191104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.191221 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.191293 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.191580 (+ 287us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.191684 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:45.191847 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:45.192006 11592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.182907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:60483)
0928 17:56:45.183087 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.183125 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.183629 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.183910 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.183914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.184502 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.184699 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.184704 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.185429 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.185430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.186237 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.186239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.186641 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.186643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.186765 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.187300 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.187308 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.188768 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.190481 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.190489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.190492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.190722 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.190924 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.190924 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.190925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.191038 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.191284 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.191290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.191450 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.191558 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.191720 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260928 17:56:45.192121 5305 external_mini_cluster.cc:1428] Setting key 43eee03198583f5495f9179ea76240f1
I20260928 17:56:45.192718 11590 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:45.192845 11590 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:45.193069 11590 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
I20260928 17:56:45.193602 11399 ts_manager.cc:194] Registered new tserver with Master: 0e88892e17db468c992a1ce180ed66be (127.5.46.65:36481)
I20260928 17:56:45.194207 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:60483
WARNING: no policy specified for oryx/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
I20260928 17:56:45.219794 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:45.303714 11595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:45.303972 11595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:45.303998 11595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:45.305255 11595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:45.305306 11595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:45.305333 11595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:45.306785 11595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:45.306988 11595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:45.307176 11595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:45.312350 11601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.312371 11603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.312404 11600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.312479 11595 server_base.cc:1034] running on GCE node
I20260928 17:56:45.312788 11595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.313040 11595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.314260 11595 hybrid_clock.cc:648] HybridClock initialized: now 1790618205314238 us; error 38 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.316941 11595 init.cc:377] Logged in from keytab as oryx/127.5.46.66@KRBTEST.COM (short username oryx)
I20260928 17:56:45.317418 11595 webserver.cc:460] Webserver started at http://127.5.46.66:32795/ using document root <none> and password file <none>
I20260928 17:56:45.317603 11595 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.317643 11595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.317734 11595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:45.318785 11595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "e098cf06d64d45268f067a6413455313"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "56308fa13dc6758ac3c19ee61499e68c"
server_key_iv: "919f956431c33c2ce4f7ca65ff483182"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.319088 11595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "e098cf06d64d45268f067a6413455313"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "56308fa13dc6758ac3c19ee61499e68c"
server_key_iv: "919f956431c33c2ce4f7ca65ff483182"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.320420 11595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:45.321101 11610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.321283 11595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:45.321386 11595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "e098cf06d64d45268f067a6413455313"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "56308fa13dc6758ac3c19ee61499e68c"
server_key_iv: "919f956431c33c2ce4f7ca65ff483182"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.321445 11595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.332605 11595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.334914 11595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:45.335029 11595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.335213 11595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:45.335515 11595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:45.335546 11595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.335585 11595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:45.335613 11595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.341455 11595 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:35385
I20260928 17:56:45.341535 11723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:35385 every 8 connection(s)
I20260928 17:56:45.341856 11595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:45.344964 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11595
I20260928 17:56:45.345049 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:45.345253 5305 external_mini_cluster.cc:1428] Setting key 7c1aa58b17ec5fa0e9ebb4cc3eb3cca6
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:45.352159 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.342628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:52823 (local address 127.5.46.126:43449)
0928 17:56:45.342699 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.342701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.342918 (+ 217us) server_negotiation.cc:407] Connection header received
0928 17:56:45.343447 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.343448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.343458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.343508 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.344553 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.345178 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.346490 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.346713 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.346798 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.349238 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.349240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.349241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.349249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.350696 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.351125 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.351126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.351126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.351184 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.351443 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.351444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.351445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.351561 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.351632 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.351971 (+ 339us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.352064 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:56:45.352388 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.342599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:52823)
0928 17:56:45.342784 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.342822 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.343357 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.343575 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.343578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.344269 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.344464 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.344469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.345278 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.345279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.346399 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.346402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.346806 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.346807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.346942 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.347481 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.347490 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.349140 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.350796 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.350803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.350806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.351038 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.351246 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.351247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.351247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.351368 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.351623 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.351629 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.351776 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.351949 (+ 173us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.352119 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
I20260928 17:56:45.352843 11724 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:45.352988 11724 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:45.353220 11724 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 17:56:45.353652 11399 ts_manager.cc:194] Registered new tserver with Master: e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
I20260928 17:56:45.354036 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:52823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
I20260928 17:56:45.370891 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:45.453545 11729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:45.453753 11729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:45.453776 11729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:45.454965 11729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:45.455006 11729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:45.455055 11729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:45.456614 11729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:43449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:45.456866 11729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:45.457099 11729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:45.463414 11735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.463454 11734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.463713 11737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.463876 11729 server_base.cc:1034] running on GCE node
I20260928 17:56:45.464071 11729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.464353 11729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.465526 11729 hybrid_clock.cc:648] HybridClock initialized: now 1790618205465507 us; error 33 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.468808 11729 init.cc:377] Logged in from keytab as oryx/127.5.46.67@KRBTEST.COM (short username oryx)
I20260928 17:56:45.469280 11729 webserver.cc:460] Webserver started at http://127.5.46.67:46821/ using document root <none> and password file <none>
I20260928 17:56:45.469512 11729 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.469578 11729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.469728 11729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:45.470789 11729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "64b0a6161f774f959c8d6332d6439a48"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "898dd9fb48bcb1c0eda16906b2474ba6"
server_key_iv: "a03add896dd66f036e1653c7fc1001b7"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.471169 11729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "64b0a6161f774f959c8d6332d6439a48"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "898dd9fb48bcb1c0eda16906b2474ba6"
server_key_iv: "a03add896dd66f036e1653c7fc1001b7"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.472710 11729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 17:56:45.473510 11744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.473670 11729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:45.473764 11729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "64b0a6161f774f959c8d6332d6439a48"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "898dd9fb48bcb1c0eda16906b2474ba6"
server_key_iv: "a03add896dd66f036e1653c7fc1001b7"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.473843 11729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.496902 11729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.498869 11729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:45.499033 11729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.499261 11729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:45.499564 11729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:45.499619 11729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.499662 11729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:45.499688 11729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.505761 11729 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:36317
I20260928 17:56:45.505831 11857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:36317 every 8 connection(s)
I20260928 17:56:45.506182 11729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:45.515519 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.506966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:50439 (local address 127.5.46.126:43449)
0928 17:56:45.507040 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.507042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.507239 (+ 197us) server_negotiation.cc:407] Connection header received
0928 17:56:45.507758 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.507759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.507769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.507821 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.508901 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.509381 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.510346 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.510533 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.510583 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.512722 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.512723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.512723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.512730 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.514118 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.514546 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.514547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.514548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.514610 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.514924 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.514925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.514925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.515052 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.515120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.515378 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.515451 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:56:45.515755 11860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.506921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:50439)
0928 17:56:45.507097 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.507147 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.507656 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.507934 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.507939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.508563 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.508779 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.508784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.509470 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.509471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.510274 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.510277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.510600 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.510600 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.510707 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.511194 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.511202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.512635 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.514202 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.514208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.514209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.514457 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.514706 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.514708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.514708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.514826 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.515123 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.515128 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.515255 (+ 127us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.515345 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.515504 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260928 17:56:45.516333 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11729
I20260928 17:56:45.516358 11858 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:45.516446 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:45.516458 11858 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:45.516655 11858 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
I20260928 17:56:45.516659 5305 external_mini_cluster.cc:1428] Setting key a3a7f3d162969beac78b432c986d618c
I20260928 17:56:45.517016 11399 ts_manager.cc:194] Registered new tserver with Master: 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:45.517526 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:50439
I20260928 17:56:45.517607 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.46.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.125.keytab.
Entry for principal oryx/127.5.46.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.125.keytab.
I20260928 17:56:45.546679 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:38883
--webserver_interface=127.5.46.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.125.keytab
--principal=oryx/127.5.46.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:43449,127.5.46.125:38883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:45.628125 11863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:45.628389 11863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:45.628429 11863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:45.629735 11863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:45.629802 11863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:45.629824 11863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:45.629845 11863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:45.629892 11863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:45.631407 11863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:43449,127.5.46.125:38883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.125.keytab
--principal=oryx/127.5.46.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:38883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.46.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
W20260928 17:56:45.631556 11863 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.46.126:43449,127.5.46.125:38883'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 17:56:45.631724 11863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:45.632014 11863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:45.637039 11863 server_base.cc:1034] running on GCE node
W20260928 17:56:45.636996 11868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.637006 11869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.636999 11871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.637458 11863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.637773 11863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.638919 11863 hybrid_clock.cc:648] HybridClock initialized: now 1790618205638899 us; error 38 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.641642 11863 init.cc:377] Logged in from keytab as oryx/127.5.46.125@KRBTEST.COM (short username oryx)
I20260928 17:56:45.642158 11863 webserver.cc:460] Webserver started at http://127.5.46.125:33085/ using document root <none> and password file <none>
I20260928 17:56:45.642398 11863 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.642467 11863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.642614 11863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:45.643707 11863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/instance:
uuid: "cce7cff552c94738b949f1234990cc23"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "7b9c06a3b2e06b814f31f05d2bc2fad8"
server_key_iv: "4df88c4500d896b0415432d4be20838f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.644177 11863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal/instance:
uuid: "cce7cff552c94738b949f1234990cc23"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "7b9c06a3b2e06b814f31f05d2bc2fad8"
server_key_iv: "4df88c4500d896b0415432d4be20838f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.645648 11863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:45.646492 11878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.646695 11863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:45.646790 11863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "cce7cff552c94738b949f1234990cc23"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "7b9c06a3b2e06b814f31f05d2bc2fad8"
server_key_iv: "4df88c4500d896b0415432d4be20838f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.646876 11863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.666916 11863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.669145 11863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:45.669302 11863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.673293 11863 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.125:38883
I20260928 17:56:45.673350 11930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.125:38883 every 8 connection(s)
I20260928 17:56:45.673713 11863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
I20260928 17:56:45.674630 11931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:45.676205 11931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:56:45.681879 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 11863
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.125@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:45.685834 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.676981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.5.46.126:43449)
0928 17:56:45.677046 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.677048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.677319 (+ 271us) server_negotiation.cc:407] Connection header received
0928 17:56:45.677782 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.677783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.677794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.677890 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.678491 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.678991 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.680054 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.680242 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.680291 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.682725 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.682727 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.682727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.682758 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.684410 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.684840 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.684842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.684842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.684909 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.685161 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.685162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.685162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.685314 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.685400 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.685658 (+ 258us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.685743 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 17:56:45.686050 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.676969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40702)
0928 17:56:45.677161 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.677225 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.677690 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.677949 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.677953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.678208 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.678416 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.678420 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.679067 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.679068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.679955 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.679957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.680310 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.680311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.680438 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.680985 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.680998 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.682631 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.684493 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.684499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.684501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.684735 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.684975 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.684976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.684977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.685094 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.685382 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.685387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.685533 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.685638 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.685793 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260928 17:56:45.686978 11931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.125@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
I20260928 17:56:45.691458 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.682401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.5.46.125:38883)
0928 17:56:45.682574 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.682578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.682634 (+ 56us) server_negotiation.cc:407] Connection header received
0928 17:56:45.682715 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.682718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.682733 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.682820 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.683519 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.684668 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.685418 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.685732 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.685808 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.688572 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.688579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.688580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.688587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.690184 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.690549 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.690553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.690553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.690629 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.690830 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.690831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.690831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.691040 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.691186 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.691242 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.691338 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
I20260928 17:56:45.694432 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.687298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43536)
0928 17:56:45.687490 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.687501 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.687552 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.687694 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.687695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.687835 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.687938 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.687939 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.688630 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.688631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.689285 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.689288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.689779 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.689780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.689840 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.690189 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.690194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.691545 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.693424 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.693425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.693425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.693580 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.693772 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.693773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.693773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.693825 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.694054 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.694055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.694091 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.694154 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.694249 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260928 17:56:45.694473 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.687357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.5.46.125:38883)
0928 17:56:45.687503 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.687505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.687537 (+ 32us) server_negotiation.cc:407] Connection header received
0928 17:56:45.687606 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.687607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.687617 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.687694 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.688017 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.688555 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.689371 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.689704 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.689785 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.691617 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.691618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.691618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.691624 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.693328 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.693658 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.693659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.693659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.693710 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.693893 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.693894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.693894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.693993 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.694064 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.694179 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.694298 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260928 17:56:45.695684 11931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Bootstrap starting.
I20260928 17:56:45.696316 11931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:45.696538 11931 log.cc:824] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:45.697237 11931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: No bootstrap required, opened a new log
I20260928 17:56:45.698568 11931 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } }
I20260928 17:56:45.698690 11931 raft_consensus.cc:374] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:45.698712 11931 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce7cff552c94738b949f1234990cc23, State: Initialized, Role: FOLLOWER
I20260928 17:56:45.698832 11931 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } }
I20260928 17:56:45.699097 11931 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:45.699079 11938 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } } }
I20260928 17:56:45.699198 11938 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 17:56:45.700881 11949 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:56:45.700974 11949 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:56:45.702499 11863 master_runner.cc:428] Detected that this master cce7cff552c94738b949f1234990cc23 is joining an existing cluster
I20260928 17:56:45.702550 11863 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.46.125:38883
I20260928 17:56:45.710137 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.703754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.5.46.126:43449)
0928 17:56:45.703812 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.703815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.703908 (+ 93us) server_negotiation.cc:407] Connection header received
0928 17:56:45.703984 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.703985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.703995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.704062 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.704509 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.705015 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.705752 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.705966 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.706041 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.707068 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.707069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.707069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.707077 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.708878 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709341 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.709343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.709343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.709404 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709594 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.709595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.709596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.709718 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.709790 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.709950 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.710034 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:56:45.710155 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.703596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40708)
0928 17:56:45.703837 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.703866 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.703923 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.704065 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.704066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.704298 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.704377 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.704379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.705095 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.705096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.705667 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.705668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.706058 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.706060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.706123 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.706593 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.706598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.706956 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.708971 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.708972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.708973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.709269 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.709471 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.709472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.709535 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.709780 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.709781 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.709826 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.709923 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.710034 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"spinlock_wait_cycles":9472,"thread_start_us":102,"threads_started":1}
I20260928 17:56:45.710706 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.703654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.5.46.125:38883)
0928 17:56:45.703695 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.703697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.703830 (+ 133us) server_negotiation.cc:407] Connection header received
0928 17:56:45.703982 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.703983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.703994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.704021 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.704495 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.705237 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.706259 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.706559 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.706600 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.707664 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.707665 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.707666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.707672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.709513 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709876 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.709877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.709877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.709925 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.710101 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.710101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.710102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.710222 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.710272 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.710474 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.710599 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":19}
I20260928 17:56:45.711181 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.703592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43542)
0928 17:56:45.703766 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.703776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.703908 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.704092 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.704092 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.704345 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.704407 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.704409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.705350 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.705351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.706153 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.706155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.706670 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.706671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.706728 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.707168 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.707173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.707566 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.709602 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.709603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.709798 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.709989 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.709989 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.709990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.710040 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.710314 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.710314 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.710347 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.710435 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.710539 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.126@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
I20260928 17:56:45.720486 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.713080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43548 (local address 127.5.46.125:38883)
0928 17:56:45.713122 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.713124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.713232 (+ 108us) server_negotiation.cc:407] Connection header received
0928 17:56:45.713338 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.713339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.713347 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.713391 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.713915 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.714369 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.715511 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.715688 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.715751 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.717970 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.717972 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.717972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.717980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.719283 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.719672 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.719673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.719673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.719730 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.719942 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.719943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.719944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.720054 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.720126 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.720300 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.720384 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:45.720515 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.712998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43548)
0928 17:56:45.713132 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.713175 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.713263 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.713397 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.713400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.713752 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.713826 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.713829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.714447 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.714448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.715416 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.715419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.715766 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.715767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.715896 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.716322 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.716328 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.717881 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.719370 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.719378 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.719381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.719585 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.719795 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.719796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.719796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.719893 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.720118 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.720125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.720177 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.720275 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.720402 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260928 17:56:45.721117 11399 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.46.125:38883: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cce7cff552c94738b949f1234990cc23"
member_type: NON_VOTER
last_known_addr {
host: "127.5.46.125"
port: 38883
}
attrs {
promote: true
}
}
dest_uuid: "e2f96d049b4e415cb617f0844b970f30"
cas_config_opid_index: -1
I20260928 17:56:45.721436 11399 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } }
I20260928 17:56:45.721907 11958 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cce7cff552c94738b949f1234990cc23 (127.5.46.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } }
I20260928 17:56:45.722254 11905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:45.723018 11958 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.46.125:38883
I20260928 17:56:45.723245 11905 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Refusing update from remote peer e2f96d049b4e415cb617f0844b970f30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 17:56:45.723227 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } }
I20260928 17:56:45.723310 11863 master.cc:537] Master@127.5.46.125:38883 shutting down...
I20260928 17:56:45.723320 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:45.724102 11959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 17:56:45.724961 11383 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 17:56:45.725737 11383 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 -> Peer cce7cff552c94738b949f1234990cc23 (127.5.46.125:38883): Couldn't send request to peer cce7cff552c94738b949f1234990cc23. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 17:56:45.726269 11863 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:45.726364 11863 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:45.726415 11863 tablet_replica.cc:330] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: stopping tablet replica
I20260928 17:56:45.739197 11863 master.cc:552] Master@127.5.46.125:38883 shutdown complete.
I20260928 17:56:45.739267 11863 master_runner.cc:305] Clearing existing system tablet
I20260928 17:56:45.739432 11863 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.739495 11863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.740177 11863 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260928 17:56:45.740653 11963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.740777 11863 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.740833 11863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.740857 11863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "cce7cff552c94738b949f1234990cc23"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "7b9c06a3b2e06b814f31f05d2bc2fad8"
server_key_iv: "4df88c4500d896b0415432d4be20838f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.741554 11863 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:45.743227 11863 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 17:56:45.743288 11863 log.cc:1196] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 17:56:45.743530 11863 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Deleting consensus metadata
I20260928 17:56:45.743654 11863 master_runner.cc:315] Copying system tablet from 127.5.46.126:43449
I20260928 17:56:45.744372 11863 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: tablet copy: Beginning tablet copy session from remote peer at address 127.5.46.126:43449
I20260928 17:56:45.750777 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.744768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40724 (local address 127.5.46.126:43449)
0928 17:56:45.744815 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.744817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.744946 (+ 129us) server_negotiation.cc:407] Connection header received
0928 17:56:45.745020 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.745021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.745033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.745093 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.745435 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.746024 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.746791 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.747027 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.747113 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.748242 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.748243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.748244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.748252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.749630 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.749998 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.750000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.750001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.750060 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.750249 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.750250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.750251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.750370 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.750443 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.750588 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.750673 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:45.750801 11969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.744710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40724)
0928 17:56:45.744871 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.744881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.744962 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.745099 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.745100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.745286 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.745354 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.745356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.746107 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.746109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.746701 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.746703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.747114 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.747115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.747183 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.747701 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.747706 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.748150 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.749718 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.749720 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.749721 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.749912 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.750124 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.750125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.750125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.750189 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.750433 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.750434 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.750480 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.750563 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.750676 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
I20260928 17:56:45.751108 11419 tablet_copy_service.cc:140] P e2f96d049b4e415cb617f0844b970f30: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cce7cff552c94738b949f1234990cc23 ({username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:40724)
I20260928 17:56:45.751210 11419 tablet_copy_service.cc:161] P e2f96d049b4e415cb617f0844b970f30: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cce7cff552c94738b949f1234990cc23 at {username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:40724: session id = cce7cff552c94738b949f1234990cc23-00000000000000000000000000000000
I20260928 17:56:45.751889 11419 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 17:56:45.752548 11863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 17:56:45.753592 11863 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: tablet copy: Starting download of 0 data blocks...
I20260928 17:56:45.753759 11863 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: tablet copy: Starting download of 1 WAL segments...
I20260928 17:56:45.754685 11863 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 17:56:45.755812 11419 tablet_copy_service.cc:342] P e2f96d049b4e415cb617f0844b970f30: Request end of tablet copy session cce7cff552c94738b949f1234990cc23-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:40724
I20260928 17:56:45.755911 11419 tablet_copy_service.cc:433] P e2f96d049b4e415cb617f0844b970f30: ending tablet copy session cce7cff552c94738b949f1234990cc23-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cce7cff552c94738b949f1234990cc23
I20260928 17:56:45.758257 11863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:45.759541 11970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.759639 11971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.759785 11863 server_base.cc:1034] running on GCE node
W20260928 17:56:45.760008 11973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.760126 11863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.760269 11863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.761497 11863 hybrid_clock.cc:648] HybridClock initialized: now 1790618205761474 us; error 37 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.762861 11863 init.cc:377] Logged in from keytab as oryx/127.5.46.125@KRBTEST.COM (short username oryx)
I20260928 17:56:45.763154 11863 webserver.cc:460] Webserver started at http://127.5.46.125:44599/ using document root <none> and password file <none>
I20260928 17:56:45.763288 11863 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.763326 11863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.764003 11863 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:45.764533 11980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.764649 11863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:45.764696 11863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "cce7cff552c94738b949f1234990cc23"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "7b9c06a3b2e06b814f31f05d2bc2fad8"
server_key_iv: "4df88c4500d896b0415432d4be20838f"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.764760 11863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.782370 11863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.784276 11863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.787354 11863 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.125:38883
I20260928 17:56:45.787369 12032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.125:38883 every 8 connection(s)
I20260928 17:56:45.787714 11863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
I20260928 17:56:45.788609 12033 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:56:45.789156 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Bootstrap starting.
I20260928 17:56:45.792232 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:56:45.792498 12033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Bootstrap complete.
I20260928 17:56:45.792713 12033 raft_consensus.cc:348] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } }
I20260928 17:56:45.792809 12033 raft_consensus.cc:729] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cce7cff552c94738b949f1234990cc23, State: Initialized, Role: LEARNER
I20260928 17:56:45.792850 12033 consensus_queue.cc:260] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } }
I20260928 17:56:45.793017 12033 sys_catalog.cc:564] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:45.793080 12036 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } }
I20260928 17:56:45.793213 12036 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: LEARNER
I20260928 17:56:45.794595 12048 catalog_manager.cc:1259] Loaded cluster ID: b2670d8e59524689a71ea3c6cb677f48
I20260928 17:56:45.794673 12048 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: loading cluster ID for follower catalog manager: success
I20260928 17:56:45.796111 12048 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: acquiring CA information for follower catalog manager: success
I20260928 17:56:45.796468 12048 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 17:56:45.800554 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.793386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.5.46.125:38883)
0928 17:56:45.793521 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.793523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.793533 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:45.793628 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.793629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.793639 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.793711 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.794386 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.795404 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.796165 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.796427 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.796502 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.797745 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.797746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.797746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.797754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.799332 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.799715 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.799717 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.799717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.799781 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.800003 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.800005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.800005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.800122 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.800242 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.800351 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.800447 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for oryx/127.5.46.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.124.keytab.
Entry for principal oryx/127.5.46.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.124.keytab.
I20260928 17:56:45.827603 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:40903
--webserver_interface=127.5.46.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.124.keytab
--principal=oryx/127.5.46.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:45.916081 12051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:45.916318 12051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:45.916357 12051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:45.917555 12051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:45.917641 12051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:45.917675 12051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:45.917706 12051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:45.917729 12051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:45.919289 12051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.124.keytab
--principal=oryx/127.5.46.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.46.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:45.919579 12051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:45.919811 12051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:45.925513 12057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.925470 12059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:45.925472 12056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:45.925706 12051 server_base.cc:1034] running on GCE node
I20260928 17:56:45.925949 12051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:45.926254 12051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:45.927441 12051 hybrid_clock.cc:648] HybridClock initialized: now 1790618205927408 us; error 48 us; skew 500 ppm
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:45.930186 12051 init.cc:377] Logged in from keytab as oryx/127.5.46.124@KRBTEST.COM (short username oryx)
I20260928 17:56:45.930671 12051 webserver.cc:460] Webserver started at http://127.5.46.124:34901/ using document root <none> and password file <none>
I20260928 17:56:45.930899 12051 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:45.930964 12051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:45.931118 12051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:45.932303 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/instance:
uuid: "b210328c640c4cdaa9254b9f6e1a7175"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "9307505fe8df75525e7bc10ac8c45f3c"
server_key_iv: "7cdbbe16aec7d17e35d0f0bdc5c6c546"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.932729 12051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal/instance:
uuid: "b210328c640c4cdaa9254b9f6e1a7175"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "9307505fe8df75525e7bc10ac8c45f3c"
server_key_iv: "7cdbbe16aec7d17e35d0f0bdc5c6c546"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.934085 12051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:45.934902 12066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:45.935082 12051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:45.935177 12051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "b210328c640c4cdaa9254b9f6e1a7175"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "9307505fe8df75525e7bc10ac8c45f3c"
server_key_iv: "7cdbbe16aec7d17e35d0f0bdc5c6c546"
server_key_version: "encryptionkey@0"
I20260928 17:56:45.935257 12051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:45.955739 12051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:45.958278 12051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:45.958504 12051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:45.963719 12051 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.124:40903
I20260928 17:56:45.963783 12118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.124:40903 every 8 connection(s)
I20260928 17:56:45.964133 12051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
I20260928 17:56:45.965138 12119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:45.966825 12119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.124@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:45.973322 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12051
I20260928 17:56:45.976034 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.967657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.5.46.126:43449)
0928 17:56:45.967755 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.967757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.967838 (+ 81us) server_negotiation.cc:407] Connection header received
0928 17:56:45.968401 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.968403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.968413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.968478 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.969110 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.969595 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.970530 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.970705 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.970755 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.972882 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.972883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.972884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.972890 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.974498 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.975032 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.975033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.975033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.975099 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.975353 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.975354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.975354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.975492 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.975558 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.975801 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.975919 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 17:56:45.976238 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.967545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40738)
0928 17:56:45.967714 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.967755 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.968304 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.968553 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.968556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.968799 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.969007 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.969010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.969671 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.969672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.970456 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.970458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.970772 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.970772 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.970879 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.971415 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.971425 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.972793 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.974580 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.974590 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.974592 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.974959 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.975180 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.975181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.975181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.975289 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.975562 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.975566 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.975684 (+ 118us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.975763 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.975956 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":119,"threads_started":1}
I20260928 17:56:45.976816 12119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.124@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
I20260928 17:56:45.982853 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.973839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56174 (local address 127.5.46.124:40903)
0928 17:56:45.974034 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.974040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.974067 (+ 27us) server_negotiation.cc:407] Connection header received
0928 17:56:45.974090 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.974092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.974108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.974198 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.974922 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.975959 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.976997 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.977395 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.977471 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.979826 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.979834 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.979834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.979841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.981586 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.981972 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.981976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.981976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.982040 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.982245 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.982245 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.982246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.982445 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.982593 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.982642 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.982722 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":104,"threads_started":1}
I20260928 17:56:45.984128 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.977107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43572)
0928 17:56:45.977148 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.977157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.977187 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.977351 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.977386 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.977509 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.977575 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.977576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.978271 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.978272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.978767 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.978769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.979163 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.979163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.979234 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.979664 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.979669 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.981239 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.983025 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.983025 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.983026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.983232 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.983432 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.983432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.983433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.983494 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.983768 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.983769 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.983811 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.983934 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.984035 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260928 17:56:45.984129 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.977184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.5.46.125:38883)
0928 17:56:45.977241 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.977243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.977251 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:45.977272 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.977273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.977283 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.977356 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.977636 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.978179 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.978845 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.979084 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.979163 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.981324 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.981326 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.981327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.981334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.982953 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.983311 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.983312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.983313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.983370 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.983578 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.983579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.983580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.983703 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.983776 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.983963 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.984044 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 17:56:45.984632 12119 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 17:56:45 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618205, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.124@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
I20260928 17:56:45.990583 12121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.984906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56176)
0928 17:56:45.984956 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:45.984976 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:45.985034 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:45.985138 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:45.985139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:45.985285 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:45.985319 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.985320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.985983 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.985984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.986488 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:45.986489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.986878 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.986878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:45.986913 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.987334 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:45.987339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:45.988568 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:45.989832 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.989832 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.989833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.989985 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.990138 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:45.990139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:45.990139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:45.990185 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:45.990352 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:45.990352 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:45.990383 (+ 31us) client_negotiation.cc:769] Sending connection context
0928 17:56:45.990432 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 17:56:45.990501 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 17:56:45.990635 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:45.984958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.5.46.124:40903)
0928 17:56:45.984988 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:45.984990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:45.985015 (+ 25us) server_negotiation.cc:407] Connection header received
0928 17:56:45.985081 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:45.985082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:45.985090 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:45.985135 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:45.985429 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.985903 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.986585 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:45.986811 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:45.986871 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:45.988642 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:45.988643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:45.988643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:45.988649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:45.989766 (+ 1117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.990043 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.990044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.990044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.990088 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:45.990222 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:45.990223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:45.990223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:45.990304 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:45.990347 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:45.990447 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 17:56:45.990543 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":14}
I20260928 17:56:45.992043 12119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Bootstrap starting.
I20260928 17:56:45.993052 12119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:45.993357 12119 log.cc:824] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:45.994851 12119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: No bootstrap required, opened a new log
I20260928 17:56:45.996680 12119 raft_consensus.cc:348] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } }
I20260928 17:56:45.996814 12119 raft_consensus.cc:374] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:45.996836 12119 raft_consensus.cc:729] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b210328c640c4cdaa9254b9f6e1a7175, State: Initialized, Role: FOLLOWER
I20260928 17:56:45.996932 12119 consensus_queue.cc:260] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } }
I20260928 17:56:45.997618 12124 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } } }
I20260928 17:56:45.997677 12119 sys_catalog.cc:564] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:45.997725 12124 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 17:56:45.999341 12135 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:56:45.999413 12135 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:56:46.001857 12051 master_runner.cc:428] Detected that this master b210328c640c4cdaa9254b9f6e1a7175 is joining an existing cluster
I20260928 17:56:46.001920 12051 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.46.124:40903
I20260928 17:56:46.031797 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.5.46.126:43449)
0928 17:56:46.003142 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.003144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.003596 (+ 452us) server_negotiation.cc:407] Connection header received
0928 17:56:46.003648 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.003650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.003661 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.003720 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.004129 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.005004 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006200 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006522 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006564 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.028685 (+ 22121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.028688 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.028688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.028698 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.030442 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.030926 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.030927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.030928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.030980 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031181 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.031182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.031182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.031332 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.031388 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.031591 (+ 203us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.031684 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 17:56:46.031963 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56190 (local address 127.5.46.124:40903)
0928 17:56:46.003354 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.003356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.004005 (+ 649us) server_negotiation.cc:407] Connection header received
0928 17:56:46.004101 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.004102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.004111 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.004141 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.004539 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.005318 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006353 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006664 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006703 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.029113 (+ 22410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.029115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.029115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.029123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.030804 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031167 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.031168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.031168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.031210 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031381 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.031381 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.031382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.031494 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.031533 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.031712 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.031803 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:56:46.032280 12142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40740)
0928 17:56:46.003519 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.003530 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.003588 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.003730 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.003731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.003954 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.004028 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.004029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.005099 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.005100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.006099 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.006101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006641 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.006707 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.007108 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.007112 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.028531 (+ 21419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.030544 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.030545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.030546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.030842 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.031054 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.031055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.031109 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.031437 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.031437 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.031476 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.031969 (+ 493us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.032179 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":98,"threads_started":1}
I20260928 17:56:46.033090 12141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43574)
0928 17:56:46.003235 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.003248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.003315 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.003469 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.003471 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.003711 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.003799 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.003802 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.004756 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.004757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.005670 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.005673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006204 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.006265 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.006763 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.006769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.028835 (+ 22066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.031663 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031663 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.031664 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.032189 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.032411 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.032412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.032412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.032476 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.032759 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.032760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.032810 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.032881 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.032987 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":17,"thread_start_us":75,"threads_started":1}
I20260928 17:56:46.033169 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.5.46.125:38883)
0928 17:56:46.003254 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.003256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.003298 (+ 42us) server_negotiation.cc:407] Connection header received
0928 17:56:46.003386 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.003387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.003397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.003452 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.003908 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.004656 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.005766 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006075 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006137 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.028969 (+ 22832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.028971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.028971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.029251 (+ 280us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.031566 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.032276 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.032277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.032277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.032332 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.032553 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.032554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.032555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.032686 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.032749 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.032984 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.033071 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:56:46.033334 12143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.003402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56190)
0928 17:56:46.003951 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.003960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.004029 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.004192 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.004192 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.004404 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.004457 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.004459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.005403 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.005404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.006275 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.006276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.006774 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.006775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.006841 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.028495 (+ 21654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.028503 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.028989 (+ 486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.030893 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.030893 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.030894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.031094 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.031273 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.031274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.031274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.031316 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.031573 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.031574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.031616 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.032826 (+ 1210us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.033222 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":67,"threads_started":1}
I20260928 17:56:46.034433 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11461
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618204, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.126@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
I20260928 17:56:46.040581 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:36481
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:43449
--webserver_port=38897
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:56:46.042488 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.034483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56196)
0928 17:56:46.034536 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.034549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.034597 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.034778 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.034780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.034974 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.035028 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.035030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.035958 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.035959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.036770 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.036772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.037248 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.037250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.037337 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.037781 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.037787 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.039442 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.041311 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.041312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.041313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.041584 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.041803 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.041804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.041805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.041879 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.042165 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.042166 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.042210 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.042271 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.042383 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 17:56:46.042857 12122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.034573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.5.46.124:40903)
0928 17:56:46.034617 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.034619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.034628 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:46.034662 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.034663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.034674 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.034724 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.035109 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.035840 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.036878 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.037169 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.037236 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.039533 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.039535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.039535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.039542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.041222 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.041662 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.041664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.041664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.041723 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.041956 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.041957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.041957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.042082 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.042140 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.042680 (+ 540us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.042756 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 17:56:46.043392 11399 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.46.124:40903: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175"
member_type: NON_VOTER
last_known_addr {
host: "127.5.46.124"
port: 40903
}
attrs {
promote: true
}
}
dest_uuid: "e2f96d049b4e415cb617f0844b970f30"
cas_config_opid_index: 5
I20260928 17:56:46.043670 11399 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } }
I20260928 17:56:46.044276 11959 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.045414 11959 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.46.124:40903
I20260928 17:56:46.045501 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.045585 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.045828 12051 master.cc:537] Master@127.5.46.124:40903 shutting down...
W20260928 17:56:46.046316 11386 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 -> Peer b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903): Couldn't send request to peer b210328c640c4cdaa9254b9f6e1a7175. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 17:56:46.049500 12051 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:46.049607 12051 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:46.049674 12051 tablet_replica.cc:330] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: stopping tablet replica
I20260928 17:56:46.050798 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.044384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43586)
0928 17:56:46.044645 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.044655 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.044702 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.044923 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.044924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.044926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.044967 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.044969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.048129 (+ 3160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.048131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.049481 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.049484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.050544 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.050545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.050611 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.050632 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.050683 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.050724 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260928 17:56:46.051116 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.044494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.5.46.125:38883)
0928 17:56:46.044615 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.044617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.044753 (+ 136us) server_negotiation.cc:407] Connection header received
0928 17:56:46.044777 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.044778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.044801 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.044867 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.045036 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.045782 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.049698 (+ 3916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.050350 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.050417 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.050500 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.051011 (+ 511us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.051053 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260928 17:56:46.051740 12007 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 LEARNER]: Refusing update from remote peer e2f96d049b4e415cb617f0844b970f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 17:56:46.052063 12145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 17:56:46.052717 12007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.054041 12145 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: attempting to promote NON_VOTER cce7cff552c94738b949f1234990cc23 to VOTER
I20260928 17:56:46.054092 12036 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2f96d049b4e415cb617f0844b970f30. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.054171 12036 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: LEARNER
I20260928 17:56:46.054204 12036 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: NON_VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: true } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.054242 12036 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: LEARNER
I20260928 17:56:46.054226 12145 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } }
I20260928 17:56:46.054957 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.054801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56202)
0928 17:56:46.054879 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.054929 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 17:56:46.055014 12007 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 LEARNER]: Refusing update from remote peer e2f96d049b4e415cb617f0844b970f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 17:56:46.055200 11959 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 17:56:46.055763 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.055675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56218)
0928 17:56:46.055709 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.055739 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40903: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 17:56:46.056041 11959 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cce7cff552c94738b949f1234990cc23 (127.5.46.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.056898 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.056963 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.057003 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.057039 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.057335 12007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cce7cff552c94738b949f1234990cc23 (127.5.46.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
W20260928 17:56:46.058240 11386 consensus_peers.cc:487] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 -> Peer b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903): Couldn't send request to peer b210328c640c4cdaa9254b9f6e1a7175. Status: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40903: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 17:56:46.058490 12147 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.058624 12147 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:56:46.063186 12051 master.cc:552] Master@127.5.46.124:40903 shutdown complete.
I20260928 17:56:46.063243 12051 master_runner.cc:305] Clearing existing system tablet
I20260928 17:56:46.063385 12051 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:46.063457 12051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:46.064363 12051 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 17:56:46.064953 12149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.065115 12051 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:46.065178 12051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:46.065204 12051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "b210328c640c4cdaa9254b9f6e1a7175"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "9307505fe8df75525e7bc10ac8c45f3c"
server_key_iv: "7cdbbe16aec7d17e35d0f0bdc5c6c546"
server_key_version: "encryptionkey@0"
I20260928 17:56:46.066004 12051 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:46.067673 12051 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 17:56:46.067751 12051 log.cc:1196] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 17:56:46.067981 12051 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Deleting consensus metadata
I20260928 17:56:46.068096 12051 master_runner.cc:315] Copying system tablet from 127.5.46.126:43449
I20260928 17:56:46.068701 12051 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: tablet copy: Beginning tablet copy session from remote peer at address 127.5.46.126:43449
I20260928 17:56:46.076077 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.069092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.5.46.126:43449)
0928 17:56:46.069144 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.069147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.069224 (+ 77us) server_negotiation.cc:407] Connection header received
0928 17:56:46.069309 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.069310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.069321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.069400 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.069783 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.070540 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.071599 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.071962 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.072036 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.073137 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.073139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.073139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.073147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.074837 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.075227 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.075229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.075229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.075291 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.075495 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.075496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.075497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.075631 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.075695 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.075900 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.075986 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:56:46.076121 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.069028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:40748)
0928 17:56:46.069158 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.069170 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.069240 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.069417 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.069418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.069626 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.069688 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.069690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.070643 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.070644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.071509 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.071512 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.072038 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.072039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.072100 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.072630 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.072635 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.073042 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.074926 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.074927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.074928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.075147 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.075360 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.075361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.075361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.075430 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.075697 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.075698 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.075757 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.075832 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.075969 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260928 17:56:46.076697 11419 tablet_copy_service.cc:140] P e2f96d049b4e415cb617f0844b970f30: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b210328c640c4cdaa9254b9f6e1a7175 ({username='oryx', principal='oryx/127.5.46.124@KRBTEST.COM'} at 127.0.0.1:40748)
I20260928 17:56:46.076782 11419 tablet_copy_service.cc:161] P e2f96d049b4e415cb617f0844b970f30: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b210328c640c4cdaa9254b9f6e1a7175 at {username='oryx', principal='oryx/127.5.46.124@KRBTEST.COM'} at 127.0.0.1:40748: session id = b210328c640c4cdaa9254b9f6e1a7175-00000000000000000000000000000000
I20260928 17:56:46.077433 11419 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 17:56:46.077973 12051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 17:56:46.079398 12051 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: tablet copy: Starting download of 0 data blocks...
I20260928 17:56:46.079555 12051 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: tablet copy: Starting download of 1 WAL segments...
I20260928 17:56:46.080439 12051 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 17:56:46.081389 11419 tablet_copy_service.cc:342] P e2f96d049b4e415cb617f0844b970f30: Request end of tablet copy session b210328c640c4cdaa9254b9f6e1a7175-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.46.124@KRBTEST.COM'} at 127.0.0.1:40748
I20260928 17:56:46.081442 11419 tablet_copy_service.cc:433] P e2f96d049b4e415cb617f0844b970f30: ending tablet copy session b210328c640c4cdaa9254b9f6e1a7175-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b210328c640c4cdaa9254b9f6e1a7175
I20260928 17:56:46.083807 12051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:46.085135 12156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.089100 12157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.089291 12159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:46.092049 12051 server_base.cc:1034] running on GCE node
I20260928 17:56:46.092198 12051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:46.092365 12051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:46.093506 12051 hybrid_clock.cc:648] HybridClock initialized: now 1790618206093444 us; error 80 us; skew 500 ppm
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:46.095162 12051 init.cc:377] Logged in from keytab as oryx/127.5.46.124@KRBTEST.COM (short username oryx)
I20260928 17:56:46.095430 12051 webserver.cc:460] Webserver started at http://127.5.46.124:36301/ using document root <none> and password file <none>
I20260928 17:56:46.095547 12051 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:46.095580 12051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:46.096391 12051 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:46.097028 12166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.097175 12051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.097222 12051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "b210328c640c4cdaa9254b9f6e1a7175"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "9307505fe8df75525e7bc10ac8c45f3c"
server_key_iv: "7cdbbe16aec7d17e35d0f0bdc5c6c546"
server_key_version: "encryptionkey@0"
I20260928 17:56:46.097275 12051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:46.104084 12051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:46.105933 12051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:46.108857 12051 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.124:40903
I20260928 17:56:46.109215 12051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
I20260928 17:56:46.110145 12219 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:56:46.110616 12219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Bootstrap starting.
I20260928 17:56:46.112008 12218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.124:40903 every 8 connection(s)
I20260928 17:56:46.113516 12219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:56:46.113759 12219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: Bootstrap complete.
I20260928 17:56:46.114001 12219 raft_consensus.cc:348] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } }
I20260928 17:56:46.114090 12219 raft_consensus.cc:729] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b210328c640c4cdaa9254b9f6e1a7175, State: Initialized, Role: LEARNER
I20260928 17:56:46.114126 12219 consensus_queue.cc:260] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } }
I20260928 17:56:46.114300 12219 sys_catalog.cc:564] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:46.114312 12222 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.114612 12222 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: LEARNER
I20260928 17:56:46.115782 12233 catalog_manager.cc:1259] Loaded cluster ID: b2670d8e59524689a71ea3c6cb677f48
I20260928 17:56:46.115895 12233 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: loading cluster ID for follower catalog manager: success
I20260928 17:56:46.117077 12233 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: acquiring CA information for follower catalog manager: success
I20260928 17:56:46.117385 12233 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 17:56:46.146504 12144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:46.146754 12144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:46.146798 12144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:46.148059 12144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:46.148128 12144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:46.148175 12144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:46.149753 12144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:36481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=38897
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:46.150033 12144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:46.150274 12144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:46.156433 12238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.156450 12239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.156436 12241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:46.156679 12144 server_base.cc:1034] running on GCE node
I20260928 17:56:46.156947 12144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:46.157217 12144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:46.158404 12144 hybrid_clock.cc:648] HybridClock initialized: now 1790618206158384 us; error 36 us; skew 500 ppm
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:46.161146 12144 init.cc:377] Logged in from keytab as oryx/127.5.46.65@KRBTEST.COM (short username oryx)
I20260928 17:56:46.161648 12144 webserver.cc:460] Webserver started at http://127.5.46.65:38897/ using document root <none> and password file <none>
I20260928 17:56:46.161861 12144 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:46.161916 12144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:46.163283 12144 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:46.163950 12248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.164119 12144 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:46.164211 12144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "0e88892e17db468c992a1ce180ed66be"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "69c4ca1bb272157ebfd33db48d486adb"
server_key_iv: "13d8fffe2ed9c932ed2132d700fd317f"
server_key_version: "encryptionkey@0"
I20260928 17:56:46.164636 12144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:46.177590 12144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:46.179566 12144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:46.179723 12144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:46.180006 12144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:46.180327 12144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:46.180367 12144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.180389 12144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:46.180433 12144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.186970 12144 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36481
I20260928 17:56:46.187057 12361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36481 every 8 connection(s)
I20260928 17:56:46.187405 12144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
I20260928 17:56:46.196964 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12144
I20260928 17:56:46.197129 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11595
I20260928 17:56:46.200037 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:57281 (local address 127.5.46.124:40903)
0928 17:56:46.188430 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.188432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.189072 (+ 640us) server_negotiation.cc:407] Connection header received
0928 17:56:46.189623 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.189624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.189638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.189707 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.190809 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191762 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.192621 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.193028 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193089 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.196210 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.196211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.196212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.196220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.198299 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.198845 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.198846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.198847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.198904 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.199208 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.199209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.199210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.199356 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.199442 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.199786 (+ 344us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.199908 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":52,"threads_started":1}
I20260928 17:56:46.200640 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.5.46.65:57281)
0928 17:56:46.188518 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.188528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.189476 (+ 948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.190096 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.190097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.190681 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.190738 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.190739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191839 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.191841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.192541 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.192543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.193088 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.193217 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.193908 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.193912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.196099 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.198412 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.198419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.198421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.198764 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.198987 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.198988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.198988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.199128 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.199444 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.199451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.199635 (+ 184us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.199754 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.199994 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":6,"thread_start_us":90,"threads_started":1}
I20260928 17:56:46.200788 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:38045 (local address 127.5.46.125:38883)
0928 17:56:46.188444 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.188446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.189051 (+ 605us) server_negotiation.cc:407] Connection header received
0928 17:56:46.189552 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.189554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.189566 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.189620 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.190761 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191278 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.192666 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.192961 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193029 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.197986 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.197988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.197989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.197998 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.199624 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.200022 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.200023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.200024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.200080 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.200278 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.200278 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.200279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.200404 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.200477 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.200623 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.200699 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 17:56:46.200799 12367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.5.46.65:38045)
0928 17:56:46.188534 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.188543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.189453 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.189706 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.189711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.190121 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.190640 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.190646 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191379 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.191379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.192553 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.192555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.193042 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.193217 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.193853 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.193890 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.196693 (+ 2803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.199699 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.199700 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.199700 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.199935 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.200154 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.200155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.200155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.200210 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.200469 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.200469 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.200512 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.200589 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.200694 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":287,"thread_start_us":65,"threads_started":1}
I20260928 17:56:46.201505 12362 heartbeater.cc:347] Connected to a master server at 127.5.46.124:40903
I20260928 17:56:46.201534 12364 heartbeater.cc:347] Connected to a master server at 127.5.46.125:38883
I20260928 17:56:46.201607 12364 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.201607 12362 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.201798 12364 heartbeater.cc:511] Master 127.5.46.125:38883 requested a full tablet report, sending...
I20260928 17:56:46.201805 12362 heartbeater.cc:511] Master 127.5.46.124:40903 requested a full tablet report, sending...
I20260928 17:56:46.202301 11997 ts_manager.cc:194] Registered new tserver with Master: 0e88892e17db468c992a1ce180ed66be (127.5.46.65:36481)
I20260928 17:56:46.202319 12177 ts_manager.cc:194] Registered new tserver with Master: 0e88892e17db468c992a1ce180ed66be (127.5.46.65:36481)
I20260928 17:56:46.203155 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:35385
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:43449
--webserver_port=32795
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:56:46.204054 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:33365 (local address 127.5.46.126:43449)
0928 17:56:46.188270 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.188273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.189020 (+ 747us) server_negotiation.cc:407] Connection header received
0928 17:56:46.189582 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.189584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.189597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.189630 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.190741 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191564 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193087 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.193303 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193401 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.195739 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.195741 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.195741 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.195749 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.197529 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.203191 (+ 5662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.203192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.203192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.203273 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.203546 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.203547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.203548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.203663 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.203721 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.203917 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.203995 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 17:56:46.204334 12366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.188148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:33365)
0928 17:56:46.188437 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.188484 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.189473 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.189710 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.189713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.190104 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.190638 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.190644 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.191649 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.191650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.192553 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.192555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.193403 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.193404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.193906 (+ 502us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.194327 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.194332 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.195638 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.202934 (+ 7296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.202935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.202935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.203111 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.203408 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.203409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.203409 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.203475 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.203728 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.203729 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.203777 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.203887 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.204003 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":6,"thread_start_us":86,"threads_started":1}
I20260928 17:56:46.204730 12363 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:46.204779 12363 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.204854 12363 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
I20260928 17:56:46.205132 11399 ts_manager.cc:194] Re-registered known tserver with Master: 0e88892e17db468c992a1ce180ed66be (127.5.46.65:36481)
I20260928 17:56:46.205641 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:33365
W20260928 17:56:46.288973 12370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:46.289194 12370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:46.289219 12370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:46.290431 12370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:46.290477 12370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:46.290503 12370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:46.292052 12370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:35385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=32795
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:46.292320 12370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:46.292583 12370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:46.298714 12376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.298725 12375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.298714 12378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:46.299034 12370 server_base.cc:1034] running on GCE node
I20260928 17:56:46.299268 12370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:46.299546 12370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:46.300737 12370 hybrid_clock.cc:648] HybridClock initialized: now 1790618206300710 us; error 49 us; skew 500 ppm
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:46.303566 12370 init.cc:377] Logged in from keytab as oryx/127.5.46.66@KRBTEST.COM (short username oryx)
I20260928 17:56:46.304142 12370 webserver.cc:460] Webserver started at http://127.5.46.66:32795/ using document root <none> and password file <none>
I20260928 17:56:46.304414 12370 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:46.304500 12370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:46.305900 12370 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:46.306636 12385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.306823 12370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:46.306938 12370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "e098cf06d64d45268f067a6413455313"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "56308fa13dc6758ac3c19ee61499e68c"
server_key_iv: "919f956431c33c2ce4f7ca65ff483182"
server_key_version: "encryptionkey@0"
I20260928 17:56:46.307293 12370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:46.340158 12370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:46.342442 12370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:46.342617 12370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:46.342867 12370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:46.343190 12370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:46.343240 12370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.343276 12370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:46.343302 12370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.349382 12370 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:35385
I20260928 17:56:46.349460 12498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:35385 every 8 connection(s)
I20260928 17:56:46.349833 12370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:46.359536 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12370
I20260928 17:56:46.359692 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11729
I20260928 17:56:46.361719 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:54409 (local address 127.5.46.125:38883)
0928 17:56:46.350859 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.350885 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.351286 (+ 401us) server_negotiation.cc:407] Connection header received
0928 17:56:46.351915 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.351917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.351931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.352002 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.353009 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353496 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.354804 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.355585 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355649 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.357891 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.357892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.357892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.357900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.360174 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.360730 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.360731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.360731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.360792 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.361041 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.361042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.361043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.361175 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.361248 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.361531 (+ 283us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.361617 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260928 17:56:46.362010 12503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.5.46.66:54409)
0928 17:56:46.351158 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.351194 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.351759 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.352118 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.352122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.352626 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.352888 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.352894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353591 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.353592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.354720 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.354722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.355655 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.355722 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.356167 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.356171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.357799 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.360356 (+ 2557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.360364 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.360367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.360640 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.360853 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.360854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.360855 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.360977 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.361238 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.361245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.361393 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.361512 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.361689 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":8,"thread_start_us":93,"threads_started":1}
I20260928 17:56:46.363086 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:42995 (local address 127.5.46.126:43449)
0928 17:56:46.350969 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.350971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.352020 (+ 1049us) server_negotiation.cc:407] Connection header received
0928 17:56:46.352416 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.352417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.352428 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.352479 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.352992 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353790 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.354653 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.354924 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.354992 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.359181 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.359182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.359183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.359192 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.361438 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.362346 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.362348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.362348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.362421 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.362615 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.362616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.362617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.362732 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.362789 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.362961 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.363026 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 17:56:46.363420 12504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:42995)
0928 17:56:46.351955 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.351965 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.352334 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.352481 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.352482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.352700 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.352887 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.352892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353882 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.353882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.354554 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.354557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.355002 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.355192 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.355918 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.355928 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.359070 (+ 3142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.361530 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.361531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.361531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.361757 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.362492 (+ 735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.362492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.362493 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.362552 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.362798 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.362799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.362858 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.362932 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.363049 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1302,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260928 17:56:46.363793 12500 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:46.363934 12500 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.364116 12500 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
I20260928 17:56:46.364601 11399 ts_manager.cc:194] Re-registered known tserver with Master: e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
I20260928 17:56:46.365160 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:42995
I20260928 17:56:46.365427 12501 heartbeater.cc:347] Connected to a master server at 127.5.46.125:38883
I20260928 17:56:46.365482 12501 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.365546 12501 heartbeater.cc:511] Master 127.5.46.125:38883 requested a full tablet report, sending...
I20260928 17:56:46.366933 11997 ts_manager.cc:194] Registered new tserver with Master: e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
I20260928 17:56:46.366955 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:36317
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:43449
--webserver_port=46821
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:56:46.368024 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:59509 (local address 127.5.46.124:40903)
0928 17:56:46.350940 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.350943 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.352075 (+ 1132us) server_negotiation.cc:407] Connection header received
0928 17:56:46.352143 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.352144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.352162 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.352228 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.353016 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353751 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355139 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.355384 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355496 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.358254 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.358255 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.358256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.358265 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.360205 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.367062 (+ 6857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.367064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.367064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.367132 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.367414 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.367415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.367415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.367531 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.367636 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.367768 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.367937 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260928 17:56:46.368216 12505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.350623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.5.46.66:59509)
0928 17:56:46.352016 (+ 1393us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.352027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.352089 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.352236 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.352236 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.352617 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.352888 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.352894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.353842 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.353843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.354715 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.354717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.355490 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.355491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.355561 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.356000 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.356005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.358179 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.366688 (+ 8509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.366689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.366689 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.366946 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.367272 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.367273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.367273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.367336 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.367604 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.367605 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.367658 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.367719 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.367866 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"mutex_wait_us":107,"thread_start_us":79,"threads_started":1}
I20260928 17:56:46.368571 12499 heartbeater.cc:347] Connected to a master server at 127.5.46.124:40903
I20260928 17:56:46.368647 12499 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.368723 12499 heartbeater.cc:511] Master 127.5.46.124:40903 requested a full tablet report, sending...
I20260928 17:56:46.369042 12177 ts_manager.cc:194] Registered new tserver with Master: e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
W20260928 17:56:46.450543 12506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:46.450806 12506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:46.450846 12506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:46.452092 12506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:46.452149 12506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:46.452194 12506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:46.453691 12506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:36317
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=46821
--tserver_master_addrs=127.5.46.126:43449,127.5.46.125:38883,127.5.46.124:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:46.453958 12506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:46.454188 12506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:46.460973 12512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.460981 12511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:46.460981 12514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:46.461756 12506 server_base.cc:1034] running on GCE node
I20260928 17:56:46.462100 12506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:46.462412 12506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:46.463581 12506 hybrid_clock.cc:648] HybridClock initialized: now 1790618206463566 us; error 29 us; skew 500 ppm
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:46.480082 12506 init.cc:377] Logged in from keytab as oryx/127.5.46.67@KRBTEST.COM (short username oryx)
I20260928 17:56:46.480587 12506 webserver.cc:460] Webserver started at http://127.5.46.67:46821/ using document root <none> and password file <none>
I20260928 17:56:46.480825 12506 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:46.480909 12506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:46.482559 12506 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:46.483281 12521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.483443 12506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:46.483527 12506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "64b0a6161f774f959c8d6332d6439a48"
format_stamp: "Formatted at 2026-09-28 17:56:45 on dist-test-slave-z2t3"
server_key: "898dd9fb48bcb1c0eda16906b2474ba6"
server_key_iv: "a03add896dd66f036e1653c7fc1001b7"
server_key_version: "encryptionkey@0"
I20260928 17:56:46.483934 12506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:46.503794 12506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:46.506191 12506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:46.506306 12506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:46.506503 12506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:46.506755 12506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:46.506784 12506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.506807 12506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:46.506819 12506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:46.513231 12506 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:36317
I20260928 17:56:46.513281 12634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:36317 every 8 connection(s)
I20260928 17:56:46.513741 12506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.125@KRBTEST.COM
Sep 28 17:56:46 dist-test-slave-z2t3 krb5kdc[11354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618206, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.124@KRBTEST.COM
I20260928 17:56:46.523607 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12506
I20260928 17:56:46.525184 12640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:54631)
0928 17:56:46.514773 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.514821 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.515434 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.515705 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.515710 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.516379 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.516477 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.516483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517445 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.517446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.518310 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.518312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.519098 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.519100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.519176 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.519719 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.519724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.521315 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.523331 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.523338 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.523341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.523724 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.523974 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.523975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.523975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.524125 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.524402 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.524408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.524563 (+ 155us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.524700 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.524912 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":22,"thread_start_us":122,"threads_started":1}
I20260928 17:56:46.525689 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:54631 (local address 127.5.46.126:43449)
0928 17:56:46.514899 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.514901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.514915 (+ 14us) server_negotiation.cc:407] Connection header received
0928 17:56:46.515525 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.515526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.515536 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.515593 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.516605 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517355 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518803 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.518993 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.519703 (+ 710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.521448 (+ 1745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.521450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.521450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.521460 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.523261 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.523797 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.523798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.523799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.523885 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.524185 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.524186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.524186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.524335 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.524411 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.525473 (+ 1062us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.525565 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:46.526053 12636 heartbeater.cc:347] Connected to a master server at 127.5.46.126:43449
I20260928 17:56:46.526160 12636 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.526392 12636 heartbeater.cc:511] Master 127.5.46.126:43449 requested a full tablet report, sending...
I20260928 17:56:46.526430 12037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:44703 (local address 127.5.46.125:38883)
0928 17:56:46.514973 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.514975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.515820 (+ 845us) server_negotiation.cc:407] Connection header received
0928 17:56:46.515933 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.515935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.515945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.516003 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.516592 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517096 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518390 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.518684 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518750 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.522279 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.522281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.522281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.522290 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.524065 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.525417 (+ 1352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.525418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.525419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.525480 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.525685 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.525686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.525686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.525820 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.525886 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.526218 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.526304 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260928 17:56:46.526778 11399 ts_manager.cc:194] Re-registered known tserver with Master: 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:46.527273 11399 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:54631
I20260928 17:56:46.527526 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:57103 (local address 127.5.46.124:40903)
0928 17:56:46.514929 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.514931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.515647 (+ 716us) server_negotiation.cc:407] Connection header received
0928 17:56:46.515709 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.515710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.515718 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.515776 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.516576 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517351 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518274 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.518600 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518668 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.522373 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.522374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.522374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.522383 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.524681 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.526737 (+ 2056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.526738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.526739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.526797 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.526997 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.526998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.526999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.527118 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.527158 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.527339 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.527436 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 17:56:46.527901 12639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.5.46.67:57103)
0928 17:56:46.515594 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.515605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.515655 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.515799 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.515800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.516265 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.516477 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.516482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517484 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.517485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.518182 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.518184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.518677 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.518824 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.519531 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.519542 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.522249 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.525031 (+ 2782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.525032 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.525032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.526654 (+ 1622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.526870 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.526870 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.526870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.526928 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.527200 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.527201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.527240 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.527647 (+ 407us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.527755 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"mutex_wait_us":54,"thread_start_us":128,"threads_started":1}
I20260928 17:56:46.528230 12635 heartbeater.cc:347] Connected to a master server at 127.5.46.124:40903
I20260928 17:56:46.528275 12635 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.528368 12635 heartbeater.cc:511] Master 127.5.46.124:40903 requested a full tablet report, sending...
I20260928 17:56:46.528440 12641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.514504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.5.46.67:44703)
0928 17:56:46.515760 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.515768 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.515871 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.516009 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.516010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.516308 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:46.516477 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.516483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.517195 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.517195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.518305 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.518308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.518749 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.518750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.518824 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.519557 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:46.519561 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:46.522191 (+ 2630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:46.525074 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.525075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.525076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.525311 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.525551 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:46.525552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:46.525553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:46.525611 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:46.525915 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:46.525916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:46.525964 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.526028 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.526144 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1213,"mutex_wait_us":48,"thread_start_us":90,"threads_started":1}
I20260928 17:56:46.528637 12177 ts_manager.cc:194] Registered new tserver with Master: 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:46.529214 12637 heartbeater.cc:347] Connected to a master server at 127.5.46.125:38883
I20260928 17:56:46.529264 12637 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:46.529313 12637 heartbeater.cc:511] Master 127.5.46.125:38883 requested a full tablet report, sending...
I20260928 17:56:46.529596 11997 ts_manager.cc:194] Registered new tserver with Master: 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:46.534448 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.524775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.5.46.126:43449)
0928 17:56:46.525874 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.525876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.525883 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:56:46.525902 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.525903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.525913 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.525961 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.526622 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.527398 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.528492 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.528771 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.528813 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.530792 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.530794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.530795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.530802 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.533069 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.533532 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.533534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.533534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.533591 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.533812 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.533813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.533814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.533990 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.534103 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.534260 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.534332 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":83,"threads_started":1}
I20260928 17:56:46.535912 12649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.525073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43590 (local address 127.5.46.125:38883)
0928 17:56:46.525188 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.525190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.525197 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:56:46.525218 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.525219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.525529 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.526778 (+ 1249us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.526814 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.527840 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.529366 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.529701 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.529797 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.531149 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.531151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.531151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.531157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.533896 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.534556 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.534557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.534557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.534617 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.534838 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.534839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.534839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.534984 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.535108 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.535687 (+ 579us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.535771 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260928 17:56:46.536542 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.525349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.5.46.124:40903)
0928 17:56:46.525606 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.525608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.529074 (+ 3466us) server_negotiation.cc:407] Connection header received
0928 17:56:46.529102 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.529103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.529117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.529526 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:46.530226 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.530772 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.531874 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.532286 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.532371 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.533411 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:46.533413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:46.533413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:46.533420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:46.535135 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.535483 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.535484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.535484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.535540 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:46.535766 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:46.535767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:46.535767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:46.535938 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:46.536037 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.536356 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.536441 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":70,"threads_started":1}
I20260928 17:56:46.536286 11399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:46.538208 11399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:46.547026 12656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.543092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:36317 (local address 127.0.0.1:45394)
0928 17:56:46.543410 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.543422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.543465 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.543665 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.543666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.543668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.543708 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.543709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.544713 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.544714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.545700 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.545702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.546734 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.546735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.546793 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.546798 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.546879 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.546925 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260928 17:56:46.547034 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.543278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.5.46.67:36317)
0928 17:56:46.543512 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.543517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.543527 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:46.543566 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.543567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.543580 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.543644 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.543777 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.544617 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.545788 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.546642 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.546728 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.546813 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.546899 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.546941 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
I20260928 17:56:46.547286 12659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.542890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:35385 (local address 127.0.0.1:39656)
0928 17:56:46.543195 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.543206 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.543259 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.544150 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.544151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.544153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.544193 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.544194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.545536 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.545537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.546295 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.546296 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.547076 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.547077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.547141 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.547145 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.547176 (+ 31us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.547218 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":103,"threads_started":1}
I20260928 17:56:46.547640 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.542980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.5.46.66:35385)
0928 17:56:46.543977 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.543982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.543993 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:46.544033 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.544035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.544051 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.544141 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.544266 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.545446 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.546381 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.546984 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.547409 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.547499 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.547526 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.547567 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":80,"threads_started":1}
I20260928 17:56:46.547876 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.542848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36481 (local address 127.0.0.1:55710)
0928 17:56:46.542923 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.542936 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.542981 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.543318 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.543319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.543321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.543379 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.543381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.544956 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.544957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.546085 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.546088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.547597 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.547598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.547665 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.547668 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.547736 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.547783 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 17:56:46.547825 12569 tablet_service.cc:1467] Processing CreateTablet for tablet e4a1705998b94b29b34d534b5626702d (DEFAULT_TABLE table=test-table [id=a9ff5ec68280478b8c98bcdf1ce8f24a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:46.547926 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.542940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.5.46.65:36481)
0928 17:56:46.543120 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.543126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.543137 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:46.543183 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.543185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.543203 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.543472 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.543539 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.544830 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.546195 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.547034 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.547596 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.547695 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.547755 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.547802 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260928 17:56:46.548256 12569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a1705998b94b29b34d534b5626702d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:46.548671 12433 tablet_service.cc:1467] Processing CreateTablet for tablet e4a1705998b94b29b34d534b5626702d (DEFAULT_TABLE table=test-table [id=a9ff5ec68280478b8c98bcdf1ce8f24a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:46.548785 12296 tablet_service.cc:1467] Processing CreateTablet for tablet e4a1705998b94b29b34d534b5626702d (DEFAULT_TABLE table=test-table [id=a9ff5ec68280478b8c98bcdf1ce8f24a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:46.548939 12433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a1705998b94b29b34d534b5626702d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:46.549105 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a1705998b94b29b34d534b5626702d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:46.552485 12661 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Bootstrap starting.
I20260928 17:56:46.552881 12662 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Bootstrap starting.
I20260928 17:56:46.553210 12661 tablet_bootstrap.cc:654] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:46.553313 12663 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Bootstrap starting.
I20260928 17:56:46.553580 12661 log.cc:824] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:46.553618 12662 tablet_bootstrap.cc:654] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:46.553936 12662 log.cc:824] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:46.554008 12663 tablet_bootstrap.cc:654] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:46.554292 12663 log.cc:824] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:46.554545 12661 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: No bootstrap required, opened a new log
I20260928 17:56:46.554670 12661 ts_tablet_manager.cc:1397] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:46.554867 12662 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: No bootstrap required, opened a new log
I20260928 17:56:46.554937 12662 ts_tablet_manager.cc:1397] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:46.557137 12662 raft_consensus.cc:348] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.557291 12662 raft_consensus.cc:374] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:46.557312 12662 raft_consensus.cc:729] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e88892e17db468c992a1ce180ed66be, State: Initialized, Role: FOLLOWER
I20260928 17:56:46.557466 12662 consensus_queue.cc:260] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.557749 12662 ts_tablet_manager.cc:1428] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:46.557780 12363 heartbeater.cc:503] Master 127.5.46.126:43449 was elected leader, sending a full tablet report...
I20260928 17:56:46.558736 12661 raft_consensus.cc:348] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.559067 12661 raft_consensus.cc:374] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:46.559115 12661 raft_consensus.cc:729] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e098cf06d64d45268f067a6413455313, State: Initialized, Role: FOLLOWER
I20260928 17:56:46.559257 12661 consensus_queue.cc:260] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.559396 12663 tablet_bootstrap.cc:492] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: No bootstrap required, opened a new log
I20260928 17:56:46.559482 12663 ts_tablet_manager.cc:1397] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260928 17:56:46.559578 12500 heartbeater.cc:503] Master 127.5.46.126:43449 was elected leader, sending a full tablet report...
I20260928 17:56:46.559825 12661 ts_tablet_manager.cc:1428] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260928 17:56:46.561364 12663 raft_consensus.cc:348] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.561501 12663 raft_consensus.cc:374] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:46.561522 12663 raft_consensus.cc:729] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b0a6161f774f959c8d6332d6439a48, State: Initialized, Role: FOLLOWER
I20260928 17:56:46.561632 12663 consensus_queue.cc:260] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.561862 12663 ts_tablet_manager.cc:1428] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:56:46.561920 12636 heartbeater.cc:503] Master 127.5.46.126:43449 was elected leader, sending a full tablet report...
W20260928 17:56:46.601141 12502 tablet.cc:2369] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:46.678141 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.675625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56236)
0928 17:56:46.675704 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.675720 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.675795 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.675974 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.675975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.675976 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.676025 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.676027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.676654 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.676654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.677356 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.677358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.677964 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.677965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.678008 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.678011 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.678052 (+ 41us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.678082 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 17:56:46.678140 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.675723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.5.46.124:40903)
0928 17:56:46.675787 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.675789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.675799 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:46.675877 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.675878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.675901 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.675960 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.676085 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.676587 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.677456 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.677890 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.677958 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.678019 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.678058 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.678084 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:56:46.678618 12193 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260928 17:56:46.679622 12670 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2f96d049b4e415cb617f0844b970f30. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.679731 12670 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: LEARNER
W20260928 17:56:46.688403 12365 tablet.cc:2369] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:56:46.764546 12638 tablet.cc:2369] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:46.922981 12667 raft_consensus.cc:482] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:46.923101 12667 raft_consensus.cc:504] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.923427 12667 leader_election.cc:288] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e098cf06d64d45268f067a6413455313 (127.5.46.66:35385), 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:46.926579 12658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.923746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:44097 (local address 127.5.46.66:35385)
0928 17:56:46.923820 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.923822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.923934 (+ 112us) server_negotiation.cc:407] Connection header received
0928 17:56:46.924018 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.924020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.924033 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.924094 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.924236 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.924872 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.925868 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.926228 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.926305 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.926315 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.926467 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.926504 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 17:56:46.926600 12674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.923658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:35385 (local address 127.5.46.65:44097)
0928 17:56:46.923816 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.923827 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.923948 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.924102 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.924103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.924105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.924174 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.924176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.924961 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.924963 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.925769 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.925771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.926315 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.926316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.926388 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.926392 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.926460 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.926504 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260928 17:56:46.926580 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.923940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:52747 (local address 127.5.46.67:36317)
0928 17:56:46.923980 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.923983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.924056 (+ 73us) server_negotiation.cc:407] Connection header received
0928 17:56:46.924105 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.924106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.924118 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.924172 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:46.924317 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.924832 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.925874 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.926227 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.926307 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.926316 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.926473 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.926508 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 17:56:46.926621 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.923891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:36317 (local address 127.5.46.65:52747)
0928 17:56:46.923982 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:46.923991 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:46.924052 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:46.924216 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:46.924217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:46.924219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:46.924263 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.924264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.924932 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.924933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.925775 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:46.925777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.926309 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.926310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:46.926398 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.926401 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:46.926467 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 17:56:46.926510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":40,"thread_start_us":38,"threads_started":1}
I20260928 17:56:46.926926 12453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a1705998b94b29b34d534b5626702d" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e098cf06d64d45268f067a6413455313" is_pre_election: true
I20260928 17:56:46.926972 12589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a1705998b94b29b34d534b5626702d" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b0a6161f774f959c8d6332d6439a48" is_pre_election: true
I20260928 17:56:46.927095 12453 raft_consensus.cc:2454] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 0.
I20260928 17:56:46.927105 12589 raft_consensus.cc:2454] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 0.
I20260928 17:56:46.927310 12249 leader_election.cc:302] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e88892e17db468c992a1ce180ed66be, e098cf06d64d45268f067a6413455313; no voters:
I20260928 17:56:46.927465 12667 raft_consensus.cc:2781] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:46.927539 12667 raft_consensus.cc:482] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:46.927558 12667 raft_consensus.cc:3037] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:46.928292 12667 raft_consensus.cc:504] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.928431 12667 leader_election.cc:288] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 election: Requested vote from peers e098cf06d64d45268f067a6413455313 (127.5.46.66:35385), 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317)
I20260928 17:56:46.928622 12589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a1705998b94b29b34d534b5626702d" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b0a6161f774f959c8d6332d6439a48"
I20260928 17:56:46.928608 12453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a1705998b94b29b34d534b5626702d" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e098cf06d64d45268f067a6413455313"
I20260928 17:56:46.928689 12589 raft_consensus.cc:3037] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:46.928692 12453 raft_consensus.cc:3037] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:46.929452 12453 raft_consensus.cc:2454] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 1.
I20260928 17:56:46.929654 12249 leader_election.cc:302] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e88892e17db468c992a1ce180ed66be, e098cf06d64d45268f067a6413455313; no voters:
I20260928 17:56:46.929756 12589 raft_consensus.cc:2454] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 1.
I20260928 17:56:46.929764 12667 raft_consensus.cc:2781] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:46.929876 12667 raft_consensus.cc:686] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 1 LEADER]: Becoming Leader. State: Replica: 0e88892e17db468c992a1ce180ed66be, State: Running, Role: LEADER
I20260928 17:56:46.929965 12667 consensus_queue.cc:237] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } }
I20260928 17:56:46.930721 11399 catalog_manager.cc:5495] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e88892e17db468c992a1ce180ed66be (127.5.46.65). New cstate: current_term: 1 leader_uuid: "0e88892e17db468c992a1ce180ed66be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:46.931347 11958 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: NON_VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: true } } }
I20260928 17:56:46.931452 11958 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.932152 12678 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: attempting to promote NON_VOTER b210328c640c4cdaa9254b9f6e1a7175 to VOTER
I20260928 17:56:46.932327 12678 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:46.932860 12007 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Refusing update from remote peer e2f96d049b4e415cb617f0844b970f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 17:56:46.932883 12193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 LEARNER]: Refusing update from remote peer e2f96d049b4e415cb617f0844b970f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260928 17:56:46.933096 12145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260928 17:56:46.933207 12678 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260928 17:56:46.933785 12678 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.933965 12007 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.934111 12193 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.934769 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.934846 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.934793 11959 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.934888 11959 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.934909 12678 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.934953 12678 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:46.935077 12680 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.935169 12680 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:56:46.935287 12670 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e2f96d049b4e415cb617f0844b970f30" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:46.935389 12670 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:56:46.990303 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:46.987818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.5.46.65:36481)
0928 17:56:46.987898 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:46.987900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:46.987911 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:46.987969 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:46.987970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:46.987989 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:46.988062 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:46.988221 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.988704 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.989472 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:46.989677 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:46.989747 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:46.989949 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:46.990055 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:46.990154 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:46.990173 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 17:56:46.990195 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 17:56:46.992286 12453 raft_consensus.cc:1261] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Refusing update from remote peer 0e88892e17db468c992a1ce180ed66be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:46.992306 12589 raft_consensus.cc:1261] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Refusing update from remote peer 0e88892e17db468c992a1ce180ed66be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:46.992653 12676 consensus_queue.cc:1035] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:46.992834 12667 consensus_queue.cc:1035] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:46.995429 12682 mvcc.cc:205] Tried to move back new op lower bound from 7334372175837249536 to 7334372175585615872. Current Snapshot: MvccSnapshot[applied={T|T < 7334372175837249536}]
I20260928 17:56:46.996703 12684 mvcc.cc:205] Tried to move back new op lower bound from 7334372175837249536 to 7334372175585615872. Current Snapshot: MvccSnapshot[applied={T|T < 7334372175837249536 or (T in {7334372175837249536})}]
I20260928 17:56:46.998160 11398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:46.998327 11398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:47.000309 12683 mvcc.cc:205] Tried to move back new op lower bound from 7334372175837249536 to 7334372175585615872. Current Snapshot: MvccSnapshot[applied={T|T < 7334372175837249536}]
I20260928 17:56:47.001847 11398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2f96d049b4e415cb617f0844b970f30: Sending DeleteTablet for 3 replicas of tablet e4a1705998b94b29b34d534b5626702d
I20260928 17:56:47.002159 12295 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a1705998b94b29b34d534b5626702d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:46 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:55710
I20260928 17:56:47.002247 12568 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a1705998b94b29b34d534b5626702d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:46 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:45394
I20260928 17:56:47.002411 12569 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a1705998b94b29b34d534b5626702d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:46 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:45394
I20260928 17:56:47.002163 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a1705998b94b29b34d534b5626702d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:46 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:55710
I20260928 17:56:47.002635 12433 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a1705998b94b29b34d534b5626702d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:46 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:39656
I20260928 17:56:47.002847 12693 tablet_replica.cc:330] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: stopping tablet replica
I20260928 17:56:47.002851 12691 tablet_replica.cc:330] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: stopping tablet replica
I20260928 17:56:47.002954 12691 raft_consensus.cc:2229] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:47.003058 12691 raft_consensus.cc:2258] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:47.003074 12693 raft_consensus.cc:2229] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:47.003297 12693 raft_consensus.cc:2258] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:47.003814 12693 ts_tablet_manager.cc:1905] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:47.003825 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11367
I20260928 17:56:47.003923 12691 ts_tablet_manager.cc:1905] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:47.006022 12693 ts_tablet_manager.cc:1918] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:47.006075 12691 ts_tablet_manager.cc:1918] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:47.006148 12691 log.cc:1196] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/e4a1705998b94b29b34d534b5626702d
I20260928 17:56:47.006248 12693 log.cc:1196] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/e4a1705998b94b29b34d534b5626702d
I20260928 17:56:47.006433 12691 ts_tablet_manager.cc:1939] T e4a1705998b94b29b34d534b5626702d P 0e88892e17db468c992a1ce180ed66be: Deleting consensus metadata
I20260928 17:56:47.006635 12693 ts_tablet_manager.cc:1939] T e4a1705998b94b29b34d534b5626702d P e098cf06d64d45268f067a6413455313: Deleting consensus metadata
W20260928 17:56:47.010604 12389 connection.cc:535] server connection from 127.0.0.1:39656 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 17:56:47.010638 12250 connection.cc:535] server connection from 127.0.0.1:55710 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 17:56:47.011819 12692 tablet_replica.cc:330] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: stopping tablet replica
I20260928 17:56:47.011986 12692 raft_consensus.cc:2229] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:47.012107 12692 raft_consensus.cc:2258] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:47.012727 12692 ts_tablet_manager.cc:1905] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:47.014555 12692 ts_tablet_manager.cc:1918] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:47.014642 12692 log.cc:1196] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/e4a1705998b94b29b34d534b5626702d
I20260928 17:56:47.014923 12692 ts_tablet_manager.cc:1939] T e4a1705998b94b29b34d534b5626702d P 64b0a6161f774f959c8d6332d6439a48: Deleting consensus metadata
W20260928 17:56:47.015359 12522 connection.cc:441] server connection from 127.0.0.1:45394 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45394 (request call id 2) could send its response
W20260928 17:56:47.015552 12522 connection.cc:441] server connection from 127.0.0.1:45394 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45394 (request call id 1) could send its response
I20260928 17:56:47.016526 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:47.003102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.5.46.124:40903)
0928 17:56:47.003144 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:47.003147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:47.008072 (+ 4925us) server_negotiation.cc:407] Connection header received
0928 17:56:47.008189 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:47.008190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:47.008202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:47.008233 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:47.009104 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.009901 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.011041 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.011350 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.011394 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:47.013137 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:47.013139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:47.013140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:47.013149 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:47.015211 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.015635 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.015636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.015637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.015695 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.015921 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.015922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.015923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.016079 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:47.016193 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:47.016341 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 17:56:47.016427 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:56:47.023555 12649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:47.013894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43606 (local address 127.5.46.125:38883)
0928 17:56:47.013942 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:47.013944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:47.014952 (+ 1008us) server_negotiation.cc:407] Connection header received
0928 17:56:47.015091 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:47.015092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:47.015108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:47.015139 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:47.016677 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.017441 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.018326 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.018554 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.018642 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:47.020386 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:47.020387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:47.020388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:47.020396 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:47.022271 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.022707 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.022708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.022709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.022769 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.022973 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.022974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.022975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.023138 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:47.023233 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:47.023400 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 17:56:47.023488 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:47.024093 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:47.013349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.5.46.124:40903)
0928 17:56:47.013392 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:47.013393 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:47.013699 (+ 306us) server_negotiation.cc:407] Connection header received
0928 17:56:47.014120 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:47.014122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:47.014133 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:47.014183 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:47.016649 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.017441 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.018627 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:47.018848 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:47.018899 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:47.020490 (+ 1591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:47.020492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:47.020492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:47.020499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:47.022416 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.022827 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.022828 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.022828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.022895 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:47.023394 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:47.023395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:47.023395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:47.023544 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:47.023641 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:47.023820 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 17:56:47.023973 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:56:48.002794 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.002574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:37279)
0928 17:56:48.002673 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.002748 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
W20260928 17:56:48.003110 12363 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:43449 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.003558 12701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.002315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:60685)
0928 17:56:48.003434 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.003499 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1083,"thread_start_us":83,"threads_started":1}
I20260928 17:56:48.003681 12703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.003282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:37875)
0928 17:56:48.003553 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.003634 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":70,"threads_started":1}
W20260928 17:56:48.003734 12636 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:43449 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
W20260928 17:56:48.003939 12500 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:43449 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.502133 12704 raft_consensus.cc:482] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2f96d049b4e415cb617f0844b970f30)
I20260928 17:56:48.502295 12704 raft_consensus.cc:504] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:48.502573 12704 leader_election.cc:288] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), cce7cff552c94738b949f1234990cc23 (127.5.46.125:38883)
I20260928 17:56:48.502664 12705 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2f96d049b4e415cb617f0844b970f30)
I20260928 17:56:48.502789 12705 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:48.502910 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.502663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41194)
0928 17:56:48.502813 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.502867 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":58,"threads_started":1}
W20260928 17:56:48.503039 12168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.503242 12705 leader_election.cc:288] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903)
I20260928 17:56:48.503347 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.503126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41206)
0928 17:56:48.503249 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.503304 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
W20260928 17:56:48.503471 11982 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.503612 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.503524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41212)
0928 17:56:48.503564 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.503587 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260928 17:56:48.503685 12168 leader_election.cc:334] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.503970 12711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.503820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41218)
0928 17:56:48.503898 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.503933 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260928 17:56:48.504042 11982 leader_election.cc:334] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.507190 12712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.503496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.5.46.124:40903)
0928 17:56:48.503636 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:48.503638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:48.503734 (+ 96us) server_negotiation.cc:407] Connection header received
0928 17:56:48.503865 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:48.503866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:48.503878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:48.503965 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:48.504119 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.504815 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506197 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.506792 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506865 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:48.506878 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:48.507061 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 17:56:48.507106 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260928 17:56:48.507189 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.503371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40903 (local address 127.0.0.1:56260)
0928 17:56:48.503647 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.503656 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:48.503758 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:48.504005 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:48.504006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:48.504009 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:48.504043 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:48.504044 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.504909 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.504910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:48.506086 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:48.506089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.506889 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:48.506965 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:48.506970 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:48.507044 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 17:56:48.507097 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":60,"threads_started":1}
I20260928 17:56:48.507193 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.502733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:38883 (local address 127.0.0.1:43620)
0928 17:56:48.502814 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.502826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:48.502915 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:48.503574 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:48.503575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:48.503577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:48.503633 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:48.503636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.504866 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.504866 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:48.506086 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:48.506089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.506919 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:48.506987 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:48.506991 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:48.507048 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 17:56:48.507097 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"thread_start_us":50,"threads_started":1}
I20260928 17:56:48.507526 12193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce7cff552c94738b949f1234990cc23" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b210328c640c4cdaa9254b9f6e1a7175" is_pre_election: true
I20260928 17:56:48.507189 12708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.502839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43620 (local address 127.5.46.125:38883)
0928 17:56:48.503082 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:48.503084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:48.503094 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:48.503119 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:48.503120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:48.503131 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:48.503178 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:48.503947 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.504764 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506197 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:48.506780 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:48.506865 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:48.506879 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:48.507061 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 17:56:48.507106 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":37,"threads_started":1}
I20260928 17:56:48.507696 12193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce7cff552c94738b949f1234990cc23 in term 1.
I20260928 17:56:48.507884 12007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b210328c640c4cdaa9254b9f6e1a7175" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cce7cff552c94738b949f1234990cc23" is_pre_election: true
I20260928 17:56:48.507937 11984 leader_election.cc:302] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b210328c640c4cdaa9254b9f6e1a7175, cce7cff552c94738b949f1234990cc23; no voters: e2f96d049b4e415cb617f0844b970f30
I20260928 17:56:48.508009 12007 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b210328c640c4cdaa9254b9f6e1a7175 in term 1.
I20260928 17:56:48.508098 12705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 17:56:48.508136 12705 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2f96d049b4e415cb617f0844b970f30)
I20260928 17:56:48.508159 12705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:56:48.508224 12167 leader_election.cc:302] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b210328c640c4cdaa9254b9f6e1a7175, cce7cff552c94738b949f1234990cc23; no voters: e2f96d049b4e415cb617f0844b970f30
I20260928 17:56:48.508360 12704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 17:56:48.508459 12704 raft_consensus.cc:482] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2f96d049b4e415cb617f0844b970f30)
I20260928 17:56:48.508507 12704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:56:48.509107 12705 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:48.509262 12705 leader_election.cc:288] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 election: Requested vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903)
I20260928 17:56:48.509447 12704 raft_consensus.cc:504] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:48.509487 12193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce7cff552c94738b949f1234990cc23" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b210328c640c4cdaa9254b9f6e1a7175"
I20260928 17:56:48.509579 12193 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cce7cff552c94738b949f1234990cc23 in current term 2: Already voted for candidate b210328c640c4cdaa9254b9f6e1a7175 in this term.
I20260928 17:56:48.509594 12704 leader_election.cc:288] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 election: Requested vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), cce7cff552c94738b949f1234990cc23 (127.5.46.125:38883)
I20260928 17:56:48.509761 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.509641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41250)
0928 17:56:48.509678 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.509738 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260928 17:56:48.509764 12007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b210328c640c4cdaa9254b9f6e1a7175" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cce7cff552c94738b949f1234990cc23"
I20260928 17:56:48.509835 12007 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b210328c640c4cdaa9254b9f6e1a7175 in current term 2: Already voted for candidate cce7cff552c94738b949f1234990cc23 in this term.
I20260928 17:56:48.509557 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.509445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41234)
0928 17:56:48.509497 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.509528 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 17:56:48.510178 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.510080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41262)
0928 17:56:48.510112 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.510139 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 17:56:48.510232 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:48.510129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41268)
0928 17:56:48.510180 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:48.510206 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 17:56:48.510285 11982 leader_election.cc:334] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
W20260928 17:56:48.510270 12168 leader_election.cc:334] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:48.510325 12168 leader_election.cc:302] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b210328c640c4cdaa9254b9f6e1a7175; no voters: cce7cff552c94738b949f1234990cc23, e2f96d049b4e415cb617f0844b970f30
I20260928 17:56:48.510324 11982 leader_election.cc:302] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cce7cff552c94738b949f1234990cc23; no voters: b210328c640c4cdaa9254b9f6e1a7175, e2f96d049b4e415cb617f0844b970f30
I20260928 17:56:48.510397 12705 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 17:56:48.510471 12704 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 17:56:49.003715 12714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:49.003460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:47457)
0928 17:56:49.003606 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:49.003648 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260928 17:56:49.004504 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:49.004089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:42729)
0928 17:56:49.004413 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:49.004453 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":54,"threads_started":1}
I20260928 17:56:49.004523 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:49.004297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:47325)
0928 17:56:49.004431 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:49.004478 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260928 17:56:50.004411 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.004204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:33937)
0928 17:56:50.004324 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.004367 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260928 17:56:50.005391 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.005148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:35787)
0928 17:56:50.005313 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.005353 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"spinlock_wait_cycles":27520,"thread_start_us":75,"threads_started":1}
I20260928 17:56:50.005528 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.005314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:54599)
0928 17:56:50.005447 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.005485 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":65,"threads_started":1}
I20260928 17:56:50.395551 12720 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:50.395656 12720 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:50.395915 12720 leader_election.cc:288] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903)
I20260928 17:56:50.396165 12193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce7cff552c94738b949f1234990cc23" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b210328c640c4cdaa9254b9f6e1a7175" is_pre_election: true
I20260928 17:56:50.396265 12193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce7cff552c94738b949f1234990cc23 in term 2.
I20260928 17:56:50.396317 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.396067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41510)
0928 17:56:50.396195 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.396232 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":83,"threads_started":1}
I20260928 17:56:50.396443 11984 leader_election.cc:302] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b210328c640c4cdaa9254b9f6e1a7175, cce7cff552c94738b949f1234990cc23; no voters:
I20260928 17:56:50.396544 12720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260928 17:56:50.396592 12720 raft_consensus.cc:482] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:50.396624 12720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 2 FOLLOWER]: Advancing to term 3
I20260928 17:56:50.396879 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.396767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41516)
0928 17:56:50.396812 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.396839 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 17:56:50.396950 11982 leader_election.cc:334] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:50.397454 12720 raft_consensus.cc:504] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:50.397603 12720 leader_election.cc:288] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 election: Requested vote from peers e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449), b210328c640c4cdaa9254b9f6e1a7175 (127.5.46.124:40903)
I20260928 17:56:50.397789 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.397680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41530)
0928 17:56:50.397716 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.397768 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 17:56:50.397781 12193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cce7cff552c94738b949f1234990cc23" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b210328c640c4cdaa9254b9f6e1a7175"
I20260928 17:56:50.397855 12193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 2 FOLLOWER]: Advancing to term 3
I20260928 17:56:50.398152 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.398063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41540)
0928 17:56:50.398107 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.398134 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 17:56:50.398232 11982 leader_election.cc:334] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
I20260928 17:56:50.398634 12193 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce7cff552c94738b949f1234990cc23 in term 3.
I20260928 17:56:50.398799 11984 leader_election.cc:302] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b210328c640c4cdaa9254b9f6e1a7175, cce7cff552c94738b949f1234990cc23; no voters: e2f96d049b4e415cb617f0844b970f30
I20260928 17:56:50.398895 12720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 17:56:50.398991 12720 raft_consensus.cc:686] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [term 3 LEADER]: Becoming Leader. State: Replica: cce7cff552c94738b949f1234990cc23, State: Running, Role: LEADER
I20260928 17:56:50.399067 12720 consensus_queue.cc:237] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } }
I20260928 17:56:50.399447 12723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce7cff552c94738b949f1234990cc23. Latest consensus state: current_term: 3 leader_uuid: "cce7cff552c94738b949f1234990cc23" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:50.399544 12723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:50.399731 12725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:50.400182 12725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:50.400242 12725 catalog_manager.cc:1259] Loaded cluster ID: b2670d8e59524689a71ea3c6cb677f48
I20260928 17:56:50.400334 12725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:50.400520 12725 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:50.400602 12725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Loaded TSK: 0
I20260928 17:56:50.400748 12725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:56:50.422763 11997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:50.424540 11997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:50.425609 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.425468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41562)
0928 17:56:50.425519 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.425575 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 17:56:50.425623 12193 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [term 3 FOLLOWER]: Refusing update from remote peer cce7cff552c94738b949f1234990cc23: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260928 17:56:50.426030 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.425936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.0.0.1:41568)
0928 17:56:50.425982 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.426009 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 17:56:50.425985 12723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260928 17:56:50.426115 11982 consensus_peers.cc:487] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 -> Peer e2f96d049b4e415cb617f0844b970f30 (127.5.46.126:43449): Couldn't send request to peer e2f96d049b4e415cb617f0844b970f30. Status: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 17:56:50.426649 12727 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cce7cff552c94738b949f1234990cc23. Latest consensus state: current_term: 3 leader_uuid: "cce7cff552c94738b949f1234990cc23" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:50.426765 12727 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:56:50.427217 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cce7cff552c94738b949f1234990cc23" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:50.427320 12728 mvcc.cc:205] Tried to move back new op lower bound from 7334372189901340672 to 7334372189795328000. Current Snapshot: MvccSnapshot[applied={T|T < 7334372189901340672}]
I20260928 17:56:50.427376 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:50.427407 12727 sys_catalog.cc:455] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cce7cff552c94738b949f1234990cc23" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "e2f96d049b4e415cb617f0844b970f30" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 43449 } } peers { permanent_uuid: "cce7cff552c94738b949f1234990cc23" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 38883 } attrs { promote: false } } peers { permanent_uuid: "b210328c640c4cdaa9254b9f6e1a7175" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40903 } attrs { promote: false } } }
I20260928 17:56:50.427479 12727 sys_catalog.cc:458] T 00000000000000000000000000000000 P b210328c640c4cdaa9254b9f6e1a7175 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:56:50.432709 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36481 (local address 127.0.0.1:55720)
0928 17:56:50.429339 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.429348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:50.429388 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:50.429677 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:50.429678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:50.429680 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:50.429740 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.429741 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430801 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.430802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.431894 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.431896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432450 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.432513 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432517 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:50.432569 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:50.432619 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":81,"threads_started":1}
I20260928 17:56:50.432711 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.5.46.65:36481)
0928 17:56:50.429369 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:50.429371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:50.429382 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:50.429513 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:50.429514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:50.429526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:50.429850 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:50.429906 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430708 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.431961 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432367 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432452 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432466 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:50.432578 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 17:56:50.432621 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":58,"threads_started":1}
I20260928 17:56:50.432899 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:36317 (local address 127.0.0.1:45408)
0928 17:56:50.429264 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.429275 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:50.429327 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:50.429958 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:50.429958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:50.429961 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:50.429996 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.429997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430845 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.430846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.431695 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.431698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432540 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.432633 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432636 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:50.432694 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 17:56:50.432743 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":27}
I20260928 17:56:50.432981 12296 tablet_service.cc:1467] Processing CreateTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a (DEFAULT_TABLE table=test-table [id=74dfd1fc0db34fc8b32e6825ebaff572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:50.433121 12296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d8501d64ac4e5c9523eda5cf845b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:50.433132 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45408 (local address 127.5.46.67:36317)
0928 17:56:50.429802 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:50.429804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:50.429812 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:50.429836 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:50.429837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:50.429848 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:50.429916 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:50.430064 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430768 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.431787 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432421 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432494 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432505 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:50.432998 (+ 493us) server_negotiation.cc:299] Negotiation successful
0928 17:56:50.433048 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":64,"threads_started":1}
I20260928 17:56:50.433566 12569 tablet_service.cc:1467] Processing CreateTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a (DEFAULT_TABLE table=test-table [id=74dfd1fc0db34fc8b32e6825ebaff572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:50.433665 12569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d8501d64ac4e5c9523eda5cf845b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:50.434453 12737 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: Bootstrap starting.
I20260928 17:56:50.434816 12737 tablet_bootstrap.cc:654] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:50.434868 12735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39666 (local address 127.5.46.66:35385)
0928 17:56:50.429467 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:50.429469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:50.429798 (+ 329us) server_negotiation.cc:407] Connection header received
0928 17:56:50.429824 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:50.429826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:50.429835 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:50.429912 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:50.430040 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430518 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.431888 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432445 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432503 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432515 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:50.432780 (+ 265us) server_negotiation.cc:299] Negotiation successful
0928 17:56:50.434779 (+ 1999us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":56,"threads_started":1}
I20260928 17:56:50.435046 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.429179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:35385 (local address 127.0.0.1:39666)
0928 17:56:50.429691 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.429699 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:50.429744 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:50.429912 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:50.429913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:50.429915 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:50.429962 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.429963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.430604 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.430605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.431768 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:50.431770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.432542 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.432543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:50.432629 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.432633 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:50.432694 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:56:50.432749 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
I20260928 17:56:50.435290 12738 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Bootstrap starting.
I20260928 17:56:50.435415 12433 tablet_service.cc:1467] Processing CreateTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a (DEFAULT_TABLE table=test-table [id=74dfd1fc0db34fc8b32e6825ebaff572]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:50.435439 12737 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: No bootstrap required, opened a new log
I20260928 17:56:50.435497 12737 ts_tablet_manager.cc:1397] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:50.435532 12433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6d8501d64ac4e5c9523eda5cf845b0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:50.435649 12738 tablet_bootstrap.cc:654] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:50.435616 12737 raft_consensus.cc:348] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.435684 12737 raft_consensus.cc:374] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:50.435704 12737 raft_consensus.cc:729] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e88892e17db468c992a1ce180ed66be, State: Initialized, Role: FOLLOWER
I20260928 17:56:50.435752 12737 consensus_queue.cc:260] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.435954 12737 ts_tablet_manager.cc:1428] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:50.436285 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.436173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:39787)
0928 17:56:50.436216 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.436257 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260928 17:56:50.436627 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.436545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:56717)
0928 17:56:50.436578 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.436600 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260928 17:56:50.437129 12738 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: No bootstrap required, opened a new log
I20260928 17:56:50.437160 12742 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Bootstrap starting.
I20260928 17:56:50.437179 12738 ts_tablet_manager.cc:1397] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 17:56:50.437318 12738 raft_consensus.cc:348] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.437376 12738 raft_consensus.cc:374] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:50.437400 12738 raft_consensus.cc:729] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64b0a6161f774f959c8d6332d6439a48, State: Initialized, Role: FOLLOWER
I20260928 17:56:50.437456 12738 consensus_queue.cc:260] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.437489 12742 tablet_bootstrap.cc:654] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:50.437674 12738 ts_tablet_manager.cc:1428] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:50.438086 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.437940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:56565)
0928 17:56:50.438022 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.438053 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260928 17:56:50.438202 12742 tablet_bootstrap.cc:492] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: No bootstrap required, opened a new log
I20260928 17:56:50.438294 12742 ts_tablet_manager.cc:1397] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:50.438454 12742 raft_consensus.cc:348] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.438553 12742 raft_consensus.cc:374] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:50.438580 12742 raft_consensus.cc:729] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e098cf06d64d45268f067a6413455313, State: Initialized, Role: FOLLOWER
I20260928 17:56:50.438627 12742 consensus_queue.cc:260] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.438799 12742 ts_tablet_manager.cc:1428] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:50.439112 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.439021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:41115)
0928 17:56:50.439057 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.439087 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260928 17:56:50.439298 12501 heartbeater.cc:503] Master 127.5.46.125:38883 was elected leader, sending a full tablet report...
I20260928 17:56:50.439433 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.439329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:55803)
0928 17:56:50.439378 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.439412 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 17:56:50.481419 12741 raft_consensus.cc:482] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:50.481520 12741 raft_consensus.cc:504] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.481653 12741 leader_election.cc:288] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317), e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
I20260928 17:56:50.481932 12589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d8501d64ac4e5c9523eda5cf845b0a" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b0a6161f774f959c8d6332d6439a48" is_pre_election: true
I20260928 17:56:50.481937 12453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d8501d64ac4e5c9523eda5cf845b0a" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e098cf06d64d45268f067a6413455313" is_pre_election: true
I20260928 17:56:50.482055 12453 raft_consensus.cc:2454] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 0.
I20260928 17:56:50.482061 12589 raft_consensus.cc:2454] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 0.
I20260928 17:56:50.482226 12249 leader_election.cc:302] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e88892e17db468c992a1ce180ed66be, 64b0a6161f774f959c8d6332d6439a48; no voters:
I20260928 17:56:50.482378 12741 raft_consensus.cc:2781] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:50.482424 12741 raft_consensus.cc:482] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:50.482438 12741 raft_consensus.cc:3037] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:50.483137 12741 raft_consensus.cc:504] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.483335 12741 leader_election.cc:288] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 election: Requested vote from peers 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317), e098cf06d64d45268f067a6413455313 (127.5.46.66:35385)
I20260928 17:56:50.483462 12589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d8501d64ac4e5c9523eda5cf845b0a" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64b0a6161f774f959c8d6332d6439a48"
I20260928 17:56:50.483475 12453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6d8501d64ac4e5c9523eda5cf845b0a" candidate_uuid: "0e88892e17db468c992a1ce180ed66be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e098cf06d64d45268f067a6413455313"
I20260928 17:56:50.483544 12589 raft_consensus.cc:3037] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:50.483551 12453 raft_consensus.cc:3037] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:50.484268 12453 raft_consensus.cc:2454] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 1.
I20260928 17:56:50.484283 12589 raft_consensus.cc:2454] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e88892e17db468c992a1ce180ed66be in term 1.
I20260928 17:56:50.484439 12249 leader_election.cc:302] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e88892e17db468c992a1ce180ed66be, e098cf06d64d45268f067a6413455313; no voters:
I20260928 17:56:50.484539 12741 raft_consensus.cc:2781] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:50.484683 12741 raft_consensus.cc:686] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 1 LEADER]: Becoming Leader. State: Replica: 0e88892e17db468c992a1ce180ed66be, State: Running, Role: LEADER
I20260928 17:56:50.484731 12741 consensus_queue.cc:237] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } }
I20260928 17:56:50.484835 12364 heartbeater.cc:503] Master 127.5.46.125:38883 was elected leader, sending a full tablet report...
I20260928 17:56:50.485146 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.485026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:53683)
0928 17:56:50.485080 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.485118 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 17:56:50.485602 11997 catalog_manager.cc:5495] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e88892e17db468c992a1ce180ed66be (127.5.46.65). New cstate: current_term: 1 leader_uuid: "0e88892e17db468c992a1ce180ed66be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e88892e17db468c992a1ce180ed66be" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:50.490816 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.488673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.5.46.65:36481)
0928 17:56:50.488712 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:50.488714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:50.488722 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:50.488767 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:50.488768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:50.488780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:50.488835 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:50.488939 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.489412 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.490146 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:50.490354 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:50.490425 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:50.490572 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:50.490606 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:50.490654 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:50.490716 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 17:56:50.490737 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 17:56:50.491695 12589 raft_consensus.cc:1261] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Refusing update from remote peer 0e88892e17db468c992a1ce180ed66be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:50.491696 12453 raft_consensus.cc:1261] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Refusing update from remote peer 0e88892e17db468c992a1ce180ed66be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:50.491941 12741 consensus_queue.cc:1035] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [LEADER]: Connected to new peer: Peer: permanent_uuid: "e098cf06d64d45268f067a6413455313" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 35385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:50.492043 12746 consensus_queue.cc:1035] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [LEADER]: Connected to new peer: Peer: permanent_uuid: "64b0a6161f774f959c8d6332d6439a48" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:50.492513 12637 heartbeater.cc:503] Master 127.5.46.125:38883 was elected leader, sending a full tablet report...
I20260928 17:56:50.492777 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.492634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:43137)
0928 17:56:50.492700 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.492743 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 17:56:50.492864 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.492761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:48471)
0928 17:56:50.492813 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.492838 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 17:56:50.494092 12717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.493997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.65:43913)
0928 17:56:50.494037 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.494063 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260928 17:56:50.494560 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.494458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.66:53553)
0928 17:56:50.494500 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.494526 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 17:56:50.495065 12718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.494968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:43449 (local address 127.5.46.67:59041)
0928 17:56:50.495006 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.495030 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:43449: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260928 17:56:50.496223 11997 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:50.496299 11997 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43606:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:50.497465 11997 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cce7cff552c94738b949f1234990cc23: Sending DeleteTablet for 3 replicas of tablet a6d8501d64ac4e5c9523eda5cf845b0a
I20260928 17:56:50.497730 12296 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:50 UTC) from {username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:55720
I20260928 17:56:50.497807 12569 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:50 UTC) from {username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:45408
I20260928 17:56:50.497819 12433 tablet_service.cc:1514] Processing DeleteTablet for tablet a6d8501d64ac4e5c9523eda5cf845b0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:50 UTC) from {username='oryx', principal='oryx/127.5.46.125@KRBTEST.COM'} at 127.0.0.1:39666
I20260928 17:56:50.497999 12758 tablet_replica.cc:330] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be: stopping tablet replica
I20260928 17:56:50.498027 12760 tablet_replica.cc:330] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: stopping tablet replica
I20260928 17:56:50.498066 12758 raft_consensus.cc:2229] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:50.498135 12758 raft_consensus.cc:2258] T a6d8501d64ac4e5c9523eda5cf845b0a P 0e88892e17db468c992a1ce180ed66be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:50.498126 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12144
I20260928 17:56:50.498077 12760 raft_consensus.cc:2229] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:50.498855 12760 raft_consensus.cc:2258] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:50.498991 12759 tablet_replica.cc:330] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: stopping tablet replica
I20260928 17:56:50.499058 12759 raft_consensus.cc:2229] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:50.499123 12759 raft_consensus.cc:2258] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:50.499243 12760 ts_tablet_manager.cc:1905] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:50.499416 12759 ts_tablet_manager.cc:1905] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:50.501070 12759 ts_tablet_manager.cc:1918] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:50.501143 12759 log.cc:1196] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/a6d8501d64ac4e5c9523eda5cf845b0a
I20260928 17:56:50.501295 12760 ts_tablet_manager.cc:1918] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:50.501366 12760 log.cc:1196] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/a6d8501d64ac4e5c9523eda5cf845b0a
I20260928 17:56:50.501412 12759 ts_tablet_manager.cc:1939] T a6d8501d64ac4e5c9523eda5cf845b0a P 64b0a6161f774f959c8d6332d6439a48: Deleting consensus metadata
I20260928 17:56:50.501657 12760 ts_tablet_manager.cc:1939] T a6d8501d64ac4e5c9523eda5cf845b0a P e098cf06d64d45268f067a6413455313: Deleting consensus metadata
I20260928 17:56:50.501750 11981 catalog_manager.cc:4841] TS 64b0a6161f774f959c8d6332d6439a48 (127.5.46.67:36317): tablet a6d8501d64ac4e5c9523eda5cf845b0a (table test-table [id=74dfd1fc0db34fc8b32e6825ebaff572]) successfully deleted
I20260928 17:56:50.501987 11981 catalog_manager.cc:4841] TS e098cf06d64d45268f067a6413455313 (127.5.46.66:35385): tablet a6d8501d64ac4e5c9523eda5cf845b0a (table test-table [id=74dfd1fc0db34fc8b32e6825ebaff572]) successfully deleted
I20260928 17:56:50.505988 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:50.505807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36481 (local address 127.0.0.1:55736)
0928 17:56:50.505881 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:50.505939 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 17:56:50.506026 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12370
W20260928 17:56:50.506210 11983 catalog_manager.cc:4584] TS 0e88892e17db468c992a1ce180ed66be (127.5.46.65:36481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6d8501d64ac4e5c9523eda5cf845b0a: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36481: connect: Connection refused (error 111)
I20260928 17:56:50.511605 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12506
I20260928 17:56:50.516902 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 11863
I20260928 17:56:50.522145 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12051
2026-09-28T17:56:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6864 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:50 dist-test-slave-z2t3 krb5kdc[12763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:50 dist-test-slave-z2t3 krb5kdc[12763](info): set up 2 sockets
Sep 28 17:56:50 dist-test-slave-z2t3 krb5kdc[12763](info): commencing operation
krb5kdc: starting...
W20260928 17:56:51.564594 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:51 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab.
Entry for principal oryx/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab.
I20260928 17:56:51.692399 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34833
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab
--principal=oryx/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:34833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:51.773823 12776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:51.774088 12776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:51.774127 12776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:51.775323 12776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:51.775394 12776 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 17:56:51.775415 12776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:51.775436 12776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:51.775466 12776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:51.775490 12776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:51.777091 12776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:34833
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.126.keytab
--principal=oryx/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:34833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:51.777390 12776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:51.777626 12776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:51.783149 12781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:51.783263 12782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:51.783209 12784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:51.783161 12776 server_base.cc:1034] running on GCE node
I20260928 17:56:51.783633 12776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:51.783951 12776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:51.785123 12776 hybrid_clock.cc:648] HybridClock initialized: now 1790618211785091 us; error 47 us; skew 500 ppm
Sep 28 17:56:51 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:51.800074 12776 init.cc:377] Logged in from keytab as oryx/127.5.46.126@KRBTEST.COM (short username oryx)
I20260928 17:56:51.800675 12776 webserver.cc:460] Webserver started at http://127.5.46.126:35745/ using document root <none> and password file <none>
I20260928 17:56:51.800949 12776 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:51.801021 12776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:51.801180 12776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:51.802342 12776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "99b3c955a6b945de99a30ed5f0e611fa"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "3e4a0c15fcb0668e54c4a54286649abc"
server_key_iv: "37e15cb162b2527c93fe28270af8b7e3"
server_key_version: "encryptionkey@0"
I20260928 17:56:51.802708 12776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "99b3c955a6b945de99a30ed5f0e611fa"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "3e4a0c15fcb0668e54c4a54286649abc"
server_key_iv: "37e15cb162b2527c93fe28270af8b7e3"
server_key_version: "encryptionkey@0"
I20260928 17:56:51.804188 12776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:51.805112 12791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:51.805354 12776 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:51.805470 12776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "99b3c955a6b945de99a30ed5f0e611fa"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "3e4a0c15fcb0668e54c4a54286649abc"
server_key_iv: "37e15cb162b2527c93fe28270af8b7e3"
server_key_version: "encryptionkey@0"
I20260928 17:56:51.805554 12776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:51.830140 12776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:51.832350 12776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:51.832526 12776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:51.837738 12776 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:34833
I20260928 17:56:51.837870 12853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:34833 every 8 connection(s)
I20260928 17:56:51.838181 12776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:51.838841 12854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:51.841214 12854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Bootstrap starting.
I20260928 17:56:51.841800 12854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:51.842054 12854 log.cc:824] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:51.842734 12854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: No bootstrap required, opened a new log
I20260928 17:56:51.844894 12854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } }
I20260928 17:56:51.845041 12854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:51.845064 12854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b3c955a6b945de99a30ed5f0e611fa, State: Initialized, Role: FOLLOWER
I20260928 17:56:51.845211 12854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } }
I20260928 17:56:51.845297 12854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:51.845322 12854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:51.845391 12854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:51.845929 12854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } }
I20260928 17:56:51.846017 12854 leader_election.cc:302] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99b3c955a6b945de99a30ed5f0e611fa; no voters:
I20260928 17:56:51.846154 12854 leader_election.cc:288] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:51.846233 12857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:51.846446 12854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:51.846452 12857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [term 1 LEADER]: Becoming Leader. State: Replica: 99b3c955a6b945de99a30ed5f0e611fa, State: Running, Role: LEADER
I20260928 17:56:51.846626 12857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } }
I20260928 17:56:51.847014 12858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99b3c955a6b945de99a30ed5f0e611fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } } }
I20260928 17:56:51.847111 12858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:51.847296 12866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:51.847343 12859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99b3c955a6b945de99a30ed5f0e611fa. Latest consensus state: current_term: 1 leader_uuid: "99b3c955a6b945de99a30ed5f0e611fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99b3c955a6b945de99a30ed5f0e611fa" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 34833 } } }
I20260928 17:56:51.847697 12859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:51.847802 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12776
I20260928 17:56:51.847926 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:51.848120 5305 external_mini_cluster.cc:1428] Setting key 1460263fd69a4ca47eee8f68ac4eb096
I20260928 17:56:51.848570 12866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:51.850152 12866 catalog_manager.cc:1347] Generated new cluster ID: 3c78cadb67bc488a9126ccc16f115eda
I20260928 17:56:51.850204 12866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:51 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:51.860088 12866 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:51.860576 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:51.848800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46356 (local address 127.5.46.126:34833)
0928 17:56:51.848985 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:51.848994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:51.849140 (+ 146us) server_negotiation.cc:407] Connection header received
0928 17:56:51.849568 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:51.849575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:51.849620 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:51.849794 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:51.850553 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:51.851982 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:51.853056 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:51.853694 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:51.853792 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:51.856695 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:51.856699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:51.856700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:51.856713 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:51.858737 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:51.859140 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:51.859143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:51.859143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:51.859261 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:51.859482 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:51.859482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:51.859483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:51.859781 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:51.859984 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:51.860125 (+ 141us) server_negotiation.cc:299] Negotiation successful
0928 17:56:51.860328 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260928 17:56:51.860780 12866 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:51.869336 12866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Generated new TSK 0
I20260928 17:56:51.869586 12866 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
I20260928 17:56:51.903129 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34833
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:51.986720 12880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:51.986955 12880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:51.986994 12880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:51.988365 12880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:51.988420 12880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:51.988443 12880 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:51.988463 12880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:51.990011 12880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:51.990222 12880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:51.990432 12880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:51.995755 12885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:51.996925 12880 server_base.cc:1034] running on GCE node
W20260928 17:56:51.997373 12888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:51.997453 12886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:51.997720 12880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:51.998001 12880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:51.999161 12880 hybrid_clock.cc:648] HybridClock initialized: now 1790618211999156 us; error 36 us; skew 500 ppm
Sep 28 17:56:51 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:52.001873 12880 init.cc:377] Logged in from keytab as oryx/127.5.46.65@KRBTEST.COM (short username oryx)
I20260928 17:56:52.002389 12880 webserver.cc:460] Webserver started at http://127.5.46.65:36985/ using document root <none> and password file <none>
I20260928 17:56:52.002619 12880 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:52.002682 12880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:52.002831 12880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:52.003999 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "23d9e96cb07145949cab38a565c1e080"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "bbd24f8583b126d96472a89bb5f48a85"
server_key_iv: "c5f5d58274b73726297a238fd1b7fa57"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.004370 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "23d9e96cb07145949cab38a565c1e080"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "bbd24f8583b126d96472a89bb5f48a85"
server_key_iv: "c5f5d58274b73726297a238fd1b7fa57"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.005751 12880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:52.006557 12895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.006721 12880 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:52.006834 12880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "23d9e96cb07145949cab38a565c1e080"
format_stamp: "Formatted at 2026-09-28 17:56:51 on dist-test-slave-z2t3"
server_key: "bbd24f8583b126d96472a89bb5f48a85"
server_key_iv: "c5f5d58274b73726297a238fd1b7fa57"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.006908 12880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:52.018725 12880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:52.021070 12880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:52.021251 12880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:52.021801 12880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:52.021864 12880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.021912 12880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:52.021939 12880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:52.029251 12880 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:46397
I20260928 17:56:52.029327 13010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:46397 every 8 connection(s)
I20260928 17:56:52.029719 12880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:52.031963 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.022372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:38865 (local address 127.5.46.126:34833)
0928 17:56:52.022427 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.022429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.022650 (+ 221us) server_negotiation.cc:407] Connection header received
0928 17:56:52.023190 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.023191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.023204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.023262 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.024200 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.024747 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.025775 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.026004 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.026090 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.029000 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.029002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.029002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.029014 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.030701 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.031145 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.031146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.031146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.031204 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.031399 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.031399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.031399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.031501 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.031557 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.031770 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.031889 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 17:56:52.032141 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.022370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.65:38865)
0928 17:56:52.022528 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.022567 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.023085 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.023381 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.023385 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.023781 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.024102 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.024108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.024848 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.024849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.025681 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.025684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.026102 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.026103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.026251 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.026894 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.026903 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.028889 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.030781 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.030785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.030787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.031089 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.031265 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.031266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.031266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.031362 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.031547 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.031551 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.031672 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.031749 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.031948 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
I20260928 17:56:52.032684 13011 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34833
I20260928 17:56:52.032778 13011 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:52.032965 13011 heartbeater.cc:511] Master 127.5.46.126:34833 requested a full tablet report, sending...
I20260928 17:56:52.033504 12808 ts_manager.cc:194] Registered new tserver with Master: 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397)
I20260928 17:56:52.034368 12808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:38865
I20260928 17:56:52.038425 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 12880
I20260928 17:56:52.038550 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:52.038751 5305 external_mini_cluster.cc:1428] Setting key 91f865afa99b0cf34e5882b19fdea0af
I20260928 17:56:52.039705 13017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.033747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.65:45781)
0928 17:56:52.033871 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.033882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.033939 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.034090 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.034091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.034251 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.034312 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.034313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.034993 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.034994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.035591 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.035593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.036132 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.036133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.036182 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.036627 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.036631 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.036914 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.038584 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.038585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.038586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.038791 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.039030 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.039031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.039032 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.039100 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.039403 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.039404 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.039458 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.039534 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.039626 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260928 17:56:52.039795 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.033804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:45781 (local address 127.5.46.126:34833)
0928 17:56:52.033844 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.033846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.033938 (+ 92us) server_negotiation.cc:407] Connection header received
0928 17:56:52.034010 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.034011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.034021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.034087 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.034373 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.034907 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.035684 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.036066 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.036129 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.036990 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.036991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.036991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.036997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.038482 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.038906 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.038907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.038907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.038964 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.039154 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.039155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.039155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.039269 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.039409 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.039573 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.039667 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
I20260928 17:56:52.067566 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34833
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:52.152926 13020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:52.153173 13020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:52.153203 13020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:52.154482 13020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:52.154531 13020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:52.154553 13020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:52.154572 13020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:52.156647 13020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:52.156903 13020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:52.157099 13020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:52.162680 13028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:52.162693 13026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:52.162709 13020 server_base.cc:1034] running on GCE node
W20260928 17:56:52.162680 13025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:52.163128 13020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:52.163408 13020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:52.164572 13020 hybrid_clock.cc:648] HybridClock initialized: now 1790618212164555 us; error 28 us; skew 500 ppm
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:52.167161 13020 init.cc:377] Logged in from keytab as oryx/127.5.46.66@KRBTEST.COM (short username oryx)
I20260928 17:56:52.167675 13020 webserver.cc:460] Webserver started at http://127.5.46.66:37509/ using document root <none> and password file <none>
I20260928 17:56:52.167944 13020 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:52.167992 13020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:52.168087 13020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:52.169196 13020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "db8c9698973143b4b7a1ad947d0149b0"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "512aaab87547087d484cf70c0ebfd7df"
server_key_iv: "f099753794fc6e2022ac69e71570b502"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.169535 13020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "db8c9698973143b4b7a1ad947d0149b0"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "512aaab87547087d484cf70c0ebfd7df"
server_key_iv: "f099753794fc6e2022ac69e71570b502"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.170894 13020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:56:52.171587 13035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.171742 13020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:56:52.171834 13020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "db8c9698973143b4b7a1ad947d0149b0"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "512aaab87547087d484cf70c0ebfd7df"
server_key_iv: "f099753794fc6e2022ac69e71570b502"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.171965 13020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:52.188503 13020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:52.190685 13020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:52.190804 13020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:52.191272 13020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:52.191305 13020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.191330 13020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:52.191344 13020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.198527 13020 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33989
I20260928 17:56:52.198585 13150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33989 every 8 connection(s)
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:52.198901 13020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:52.202200 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.191955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:53003 (local address 127.5.46.126:34833)
0928 17:56:52.192000 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.192002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.192286 (+ 284us) server_negotiation.cc:407] Connection header received
0928 17:56:52.192766 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.192768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.192784 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.192865 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.193776 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.194652 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.196404 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.196731 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.196816 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.199496 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.199497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.199497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.199505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.200857 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.201255 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.201256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.201257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.201310 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.201578 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.201579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.201580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.201730 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.201806 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.202047 (+ 241us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.202125 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:56:52.202301 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.191978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.66:53003)
0928 17:56:52.192148 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.192191 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.192667 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.192933 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.192937 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.193360 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.193659 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.193667 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.194763 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.194764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.196316 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.196320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.196829 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.196829 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.196961 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.197535 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.197545 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.199389 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.200931 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.200937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.200939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.201187 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.201395 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.201396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.201396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.201494 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.201799 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.201803 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.201924 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.202014 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.202138 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260928 17:56:52.203269 13151 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34833
I20260928 17:56:52.203377 13151 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:52.203408 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13020
I20260928 17:56:52.203548 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:52.203624 13151 heartbeater.cc:511] Master 127.5.46.126:34833 requested a full tablet report, sending...
I20260928 17:56:52.203817 5305 external_mini_cluster.cc:1428] Setting key 7b0080925f6d22576266dd262495fdf5
I20260928 17:56:52.204070 12808 ts_manager.cc:194] Registered new tserver with Master: db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989)
I20260928 17:56:52.204662 12808 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:53003
I20260928 17:56:52.212476 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.204497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.66:34769)
0928 17:56:52.204623 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.204633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.204680 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.206308 (+ 1628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.206309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.206463 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.206523 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.206524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.207165 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.207167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.207680 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.207682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.208151 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.208153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.208247 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.208762 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.208768 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.209206 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.211367 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.211369 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.211369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.211556 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.211765 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.211766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.211766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.211818 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.212223 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.212225 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.212267 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.212319 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.212402 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":75,"threads_started":1}
I20260928 17:56:52.212571 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.204569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:34769 (local address 127.5.46.126:34833)
0928 17:56:52.204873 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.204875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.204883 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:52.204904 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.204905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.204916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.204976 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.206607 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.207077 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.207777 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.208054 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.208137 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.209356 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.209358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.209358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.209367 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.211271 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.211634 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.211635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.211635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.211698 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.211936 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.211937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.211938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.212137 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.212212 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.212365 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.212445 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
WARNING: no policy specified for oryx/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
I20260928 17:56:52.231451 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:34833
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:52.315474 13160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:52.315711 13160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:52.315739 13160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:52.316999 13160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:52.317055 13160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:52.317135 13160 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 17:56:52.317181 13160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:52.318825 13160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.46.126:34833
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:52.319105 13160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:52.319413 13160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:52.324715 13165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:52.324750 13168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:52.324709 13166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:52.324971 13160 server_base.cc:1034] running on GCE node
I20260928 17:56:52.325227 13160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:52.325551 13160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:52.326747 13160 hybrid_clock.cc:648] HybridClock initialized: now 1790618212326729 us; error 33 us; skew 500 ppm
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:52.329458 13160 init.cc:377] Logged in from keytab as oryx/127.5.46.67@KRBTEST.COM (short username oryx)
I20260928 17:56:52.329958 13160 webserver.cc:460] Webserver started at http://127.5.46.67:34587/ using document root <none> and password file <none>
I20260928 17:56:52.330199 13160 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:52.330272 13160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:52.330417 13160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:52.331477 13160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "69971babcf6f44529935713c62644f2b"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "f5f5a296490a342592a465f803dcc7e6"
server_key_iv: "cdfeca83ef440effa2b6579969b40272"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.331826 13160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "69971babcf6f44529935713c62644f2b"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "f5f5a296490a342592a465f803dcc7e6"
server_key_iv: "cdfeca83ef440effa2b6579969b40272"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.333293 13160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:52.334182 13175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.334415 13160 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:52.334617 13160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "69971babcf6f44529935713c62644f2b"
format_stamp: "Formatted at 2026-09-28 17:56:52 on dist-test-slave-z2t3"
server_key: "f5f5a296490a342592a465f803dcc7e6"
server_key_iv: "cdfeca83ef440effa2b6579969b40272"
server_key_version: "encryptionkey@0"
I20260928 17:56:52.334692 13160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:52.346416 13160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:52.349099 13160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:52.349270 13160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:52.349896 13160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:52.349948 13160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.349983 13160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:52.350001 13160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:52.357467 13160 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:43759
I20260928 17:56:52.357579 13290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:43759 every 8 connection(s)
I20260928 17:56:52.357968 13160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:56:52.360004 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.350565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:45119 (local address 127.5.46.126:34833)
0928 17:56:52.350626 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.350629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.350671 (+ 42us) server_negotiation.cc:407] Connection header received
0928 17:56:52.351463 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.351464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.351476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.351580 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.352636 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.353155 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.354232 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.354426 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.354509 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.357074 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.357076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.357077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.357086 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.358751 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.359203 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.359204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.359204 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.359258 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.359443 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.359443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.359443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.359550 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.359600 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.359794 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.359932 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 17:56:52.360172 13184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.350353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.67:45119)
0928 17:56:52.350528 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.350580 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.351314 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.351691 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.351697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.352169 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.352515 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.352522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.353257 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.353258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.354137 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.354140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.354529 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.354530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.354665 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.355244 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.355258 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.356946 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.358819 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.358824 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.358825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.359133 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.359302 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.359302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.359302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.359396 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.359589 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.359592 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.359705 (+ 113us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.359781 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.359975 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260928 17:56:52.360620 13291 heartbeater.cc:347] Connected to a master server at 127.5.46.126:34833
I20260928 17:56:52.360702 13291 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:52.360927 13291 heartbeater.cc:511] Master 127.5.46.126:34833 requested a full tablet report, sending...
I20260928 17:56:52.361310 12807 ts_manager.cc:194] Registered new tserver with Master: 69971babcf6f44529935713c62644f2b (127.5.46.67:43759)
I20260928 17:56:52.361717 12807 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:45119
I20260928 17:56:52.367388 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13160
I20260928 17:56:52.367527 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:52.367762 5305 external_mini_cluster.cc:1428] Setting key dfdf88bc63201e0fb88e4fd229f6edcc
I20260928 17:56:52.368009 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.361954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.5.46.67:46619)
0928 17:56:52.362090 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.362105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.362146 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.362575 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.362577 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.362758 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.362822 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.362824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.363425 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.363426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.364126 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.364128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.364435 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.364436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.364486 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.364895 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.364899 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.365184 (+ 285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.366763 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.366764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.366764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.366930 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.367217 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.367224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.367224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.367290 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.367627 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.367628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.367679 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.367757 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.367907 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260928 17:56:52.368033 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.362002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:46619 (local address 127.5.46.126:34833)
0928 17:56:52.362452 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.362453 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.362461 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:52.362477 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.362477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.362485 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.362538 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.362901 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.363342 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.364190 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.364367 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.364428 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.365271 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.365272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.365273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.365279 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.366673 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.367033 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.367034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.367034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.367091 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.367364 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.367365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.367365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.367522 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.367595 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.367818 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.367921 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430}
I20260928 17:56:52.368762 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 17:56:52.390218 12876 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:52.384536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.5.46.126:34833)
0928 17:56:52.384609 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.384611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.384621 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:52.384655 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.384656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.384666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.384718 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.385278 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.385789 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.386488 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.386665 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.386718 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.390124 (+ 3406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46360: BlockingRecv error: recv got EOF from 127.0.0.1:46360 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618212, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:52.398100 12876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.391004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.5.46.126:34833)
0928 17:56:52.391068 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.391070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.391117 (+ 47us) server_negotiation.cc:407] Connection header received
0928 17:56:52.391164 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.391165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.391174 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.391222 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.391687 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.392263 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.392944 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.393132 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.393188 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.395381 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.395382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.395382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.395389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.397021 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.397503 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.397504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.397504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.397559 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.397719 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.397719 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.397719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.397830 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.397907 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.397970 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.398032 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 17:56:52.399838 12808 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:52.401662 12808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:52.408336 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33989 (local address 127.0.0.1:41618)
0928 17:56:52.404718 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.404728 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.404804 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.405053 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.405053 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.405055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.405088 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.405090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.405943 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.405944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.407192 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.407194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.407996 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.407997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.408076 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408092 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.408190 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.408248 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":75,"threads_started":1}
I20260928 17:56:52.408397 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.5.46.66:33989)
0928 17:56:52.404703 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.404712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.404866 (+ 154us) server_negotiation.cc:407] Connection header received
0928 17:56:52.404914 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.404916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.404930 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.404996 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.405144 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.405832 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.407284 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.407903 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.407995 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408073 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.408207 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.408252 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260928 17:56:52.408764 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.5.46.67:43759)
0928 17:56:52.404339 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.404344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.404582 (+ 238us) server_negotiation.cc:407] Connection header received
0928 17:56:52.404716 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.404718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.404734 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.404788 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.405047 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.406187 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.407421 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.408205 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.408279 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408356 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.408630 (+ 274us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.408676 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260928 17:56:52.408895 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56714 (local address 127.5.46.65:46397)
0928 17:56:52.404510 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.404514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.404753 (+ 239us) server_negotiation.cc:407] Connection header received
0928 17:56:52.404801 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.404803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.404820 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.404911 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.405109 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.406187 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.407478 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.408209 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.408298 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408362 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.408742 (+ 380us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.408785 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":167,"threads_started":1}
I20260928 17:56:52.408656 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46397 (local address 127.0.0.1:56714)
0928 17:56:52.404526 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.404538 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.404600 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.404980 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.404981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.404983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.405039 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.405041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.406282 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.406283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.407374 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.407393 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.408397 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.408398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.408469 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408474 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.408525 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.408573 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260928 17:56:52.408735 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.404110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.0.0.1:35948)
0928 17:56:52.404480 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.404525 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.404599 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.404900 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.404904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.404906 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.404979 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.404981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.406291 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.406292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.407339 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.407341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.408396 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.408397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.408535 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.408538 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.408599 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.408634 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260928 17:56:52.409710 13085 tablet_service.cc:1467] Processing CreateTablet for tablet 09b1797e24704be7bf63ec60ed31d860 (DEFAULT_TABLE table=test-table [id=02ec73e4334f49fa894383d097cfea68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.409711 12945 tablet_service.cc:1467] Processing CreateTablet for tablet 09b1797e24704be7bf63ec60ed31d860 (DEFAULT_TABLE table=test-table [id=02ec73e4334f49fa894383d097cfea68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.410079 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b1797e24704be7bf63ec60ed31d860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.410079 13085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b1797e24704be7bf63ec60ed31d860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.413993 13319 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Bootstrap starting.
I20260928 17:56:52.414215 13320 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Bootstrap starting.
I20260928 17:56:52.414795 13319 tablet_bootstrap.cc:654] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.414856 13320 tablet_bootstrap.cc:654] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.414958 13225 tablet_service.cc:1467] Processing CreateTablet for tablet 09b1797e24704be7bf63ec60ed31d860 (DEFAULT_TABLE table=test-table [id=02ec73e4334f49fa894383d097cfea68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.415153 13319 log.cc:824] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:52.415153 13320 log.cc:824] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:52.415307 13225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09b1797e24704be7bf63ec60ed31d860. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.416002 13319 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: No bootstrap required, opened a new log
I20260928 17:56:52.416088 13319 ts_tablet_manager.cc:1397] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:52.416208 13320 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: No bootstrap required, opened a new log
I20260928 17:56:52.416291 13320 ts_tablet_manager.cc:1397] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:52.417519 13319 raft_consensus.cc:348] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.417678 13319 raft_consensus.cc:374] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.417721 13319 raft_consensus.cc:729] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.417845 13319 consensus_queue.cc:260] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.418146 13319 ts_tablet_manager.cc:1428] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:56:52.418155 13011 heartbeater.cc:503] Master 127.5.46.126:34833 was elected leader, sending a full tablet report...
I20260928 17:56:52.418318 13320 raft_consensus.cc:348] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.418514 13320 raft_consensus.cc:374] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.418560 13320 raft_consensus.cc:729] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.418679 13320 consensus_queue.cc:260] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.418952 13320 ts_tablet_manager.cc:1428] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:56:52.419073 13151 heartbeater.cc:503] Master 127.5.46.126:34833 was elected leader, sending a full tablet report...
I20260928 17:56:52.419189 13325 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Bootstrap starting.
I20260928 17:56:52.419912 13325 tablet_bootstrap.cc:654] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.420248 13325 log.cc:824] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:52.421000 13325 tablet_bootstrap.cc:492] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: No bootstrap required, opened a new log
I20260928 17:56:52.421056 13325 ts_tablet_manager.cc:1397] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 17:56:52.422537 13325 raft_consensus.cc:348] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.422669 13325 raft_consensus.cc:374] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.422691 13325 raft_consensus.cc:729] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69971babcf6f44529935713c62644f2b, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.422827 13325 consensus_queue.cc:260] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.423086 13325 ts_tablet_manager.cc:1428] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 17:56:52.423139 13291 heartbeater.cc:503] Master 127.5.46.126:34833 was elected leader, sending a full tablet report...
W20260928 17:56:52.449529 13152 tablet.cc:2369] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:52.529455 13324 raft_consensus.cc:482] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:52.529692 13324 raft_consensus.cc:504] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.530093 13324 leader_election.cc:288] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69971babcf6f44529935713c62644f2b (127.5.46.67:43759), 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397)
W20260928 17:56:52.530735 13012 tablet.cc:2369] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:52.534297 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.530415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:47497 (local address 127.5.46.67:43759)
0928 17:56:52.530464 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.530466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.530602 (+ 136us) server_negotiation.cc:407] Connection header received
0928 17:56:52.530699 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.530700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.530713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.530744 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.531026 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.531688 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533173 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.533834 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533939 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.533953 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.534172 (+ 219us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.534215 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 17:56:52.534297 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.530371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:60183 (local address 127.5.46.65:46397)
0928 17:56:52.530443 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.530444 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.530454 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:52.530548 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.530555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.530567 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.530643 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.530818 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.531583 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533173 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.533885 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533956 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.533965 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.534172 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.534214 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 17:56:52.534399 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.530336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.5.46.66:47497)
0928 17:56:52.530518 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.530527 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.530605 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.530815 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.530815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.530817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.530935 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.530937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.531766 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.531767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.533065 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.533069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.533986 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.534078 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.534082 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.534156 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.534203 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":33,"thread_start_us":122,"threads_started":1}
I20260928 17:56:52.534415 13044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.530300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46397 (local address 127.5.46.66:60183)
0928 17:56:52.530385 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.530397 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.530450 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.530663 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.530664 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.530666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.530742 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.530744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.531666 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.531666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.533065 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.533069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.533985 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.533987 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.534078 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.534082 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.534156 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.534203 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":68}
I20260928 17:56:52.534795 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b1797e24704be7bf63ec60ed31d860" candidate_uuid: "db8c9698973143b4b7a1ad947d0149b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080" is_pre_election: true
I20260928 17:56:52.534813 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b1797e24704be7bf63ec60ed31d860" candidate_uuid: "db8c9698973143b4b7a1ad947d0149b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b" is_pre_election: true
I20260928 17:56:52.534946 12965 raft_consensus.cc:2454] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8c9698973143b4b7a1ad947d0149b0 in term 0.
I20260928 17:56:52.534981 13245 raft_consensus.cc:2454] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db8c9698973143b4b7a1ad947d0149b0 in term 0.
I20260928 17:56:52.535575 13036 leader_election.cc:302] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, db8c9698973143b4b7a1ad947d0149b0; no voters:
I20260928 17:56:52.535784 13324 raft_consensus.cc:2781] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:52.535833 13324 raft_consensus.cc:482] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:52.535904 13324 raft_consensus.cc:3037] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.536608 13324 raft_consensus.cc:504] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.536759 13324 leader_election.cc:288] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [CANDIDATE]: Term 1 election: Requested vote from peers 69971babcf6f44529935713c62644f2b (127.5.46.67:43759), 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397)
I20260928 17:56:52.536952 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b1797e24704be7bf63ec60ed31d860" candidate_uuid: "db8c9698973143b4b7a1ad947d0149b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b"
I20260928 17:56:52.536952 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09b1797e24704be7bf63ec60ed31d860" candidate_uuid: "db8c9698973143b4b7a1ad947d0149b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080"
I20260928 17:56:52.537053 13245 raft_consensus.cc:3037] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.537053 12965 raft_consensus.cc:3037] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.538041 13245 raft_consensus.cc:2454] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8c9698973143b4b7a1ad947d0149b0 in term 1.
I20260928 17:56:52.538041 12965 raft_consensus.cc:2454] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db8c9698973143b4b7a1ad947d0149b0 in term 1.
I20260928 17:56:52.538273 13036 leader_election.cc:302] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, db8c9698973143b4b7a1ad947d0149b0; no voters:
I20260928 17:56:52.538378 13324 raft_consensus.cc:2781] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:52.538544 13324 raft_consensus.cc:686] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 1 LEADER]: Becoming Leader. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Running, Role: LEADER
I20260928 17:56:52.538667 13324 consensus_queue.cc:237] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } }
I20260928 17:56:52.539615 12807 catalog_manager.cc:5495] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "db8c9698973143b4b7a1ad947d0149b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:52.553913 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.551365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.5.46.66:33989)
0928 17:56:52.551407 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.551409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.551419 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:52.551490 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.551491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.551510 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.551578 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:52.551753 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.552318 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.553116 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.553331 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.553382 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.553566 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:52.553671 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:52.553756 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.553832 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.553852 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:56:52.556439 12965 raft_consensus.cc:1261] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Refusing update from remote peer db8c9698973143b4b7a1ad947d0149b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:52.556439 13245 raft_consensus.cc:1261] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Refusing update from remote peer db8c9698973143b4b7a1ad947d0149b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:52.556788 13329 consensus_queue.cc:1035] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:52.556928 13324 consensus_queue.cc:1035] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:52.559518 13332 mvcc.cc:205] Tried to move back new op lower bound from 7334372198628139008 to 7334372198559141888. Current Snapshot: MvccSnapshot[applied={T|T < 7334372198628139008}]
I20260928 17:56:52.559957 13334 mvcc.cc:205] Tried to move back new op lower bound from 7334372198628139008 to 7334372198559141888. Current Snapshot: MvccSnapshot[applied={T|T < 7334372198628139008}]
I20260928 17:56:52.562494 12807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:52.562592 12807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:52.563541 12807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Sending DeleteTablet for 3 replicas of tablet 09b1797e24704be7bf63ec60ed31d860
I20260928 17:56:52.563901 12945 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b1797e24704be7bf63ec60ed31d860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:52 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:56714
I20260928 17:56:52.563906 13225 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b1797e24704be7bf63ec60ed31d860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:52 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35948
I20260928 17:56:52.563953 13085 tablet_service.cc:1514] Processing DeleteTablet for tablet 09b1797e24704be7bf63ec60ed31d860 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:52 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41618
I20260928 17:56:52.564107 13342 tablet_replica.cc:330] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: stopping tablet replica
I20260928 17:56:52.564206 13342 raft_consensus.cc:2229] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:52.564321 13342 raft_consensus.cc:2258] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:52.564379 13344 tablet_replica.cc:330] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: stopping tablet replica
I20260928 17:56:52.564405 12807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:52.564544 12807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:52.564549 13344 raft_consensus.cc:2229] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:52.564677 13344 raft_consensus.cc:2258] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:52.564909 13343 tablet_replica.cc:330] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: stopping tablet replica
I20260928 17:56:52.564808 13342 ts_tablet_manager.cc:1905] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:52.565001 13343 raft_consensus.cc:2229] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:52.565366 13343 raft_consensus.cc:2258] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:52.565692 13344 ts_tablet_manager.cc:1905] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:52.566030 12945 tablet_service.cc:1467] Processing CreateTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 (DEFAULT_TABLE table=test-table [id=45dd3f609fa745858e1a71dd64605f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.566144 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b4335984774af0b7b3fc5ce4132c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.566673 13342 ts_tablet_manager.cc:1918] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:52.566741 13342 log.cc:1196] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/09b1797e24704be7bf63ec60ed31d860
I20260928 17:56:52.566991 13342 ts_tablet_manager.cc:1939] T 09b1797e24704be7bf63ec60ed31d860 P 23d9e96cb07145949cab38a565c1e080: Deleting consensus metadata
I20260928 17:56:52.567368 13319 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Bootstrap starting.
I20260928 17:56:52.567413 13344 ts_tablet_manager.cc:1918] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:52.567468 13344 log.cc:1196] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/09b1797e24704be7bf63ec60ed31d860
I20260928 17:56:52.567747 13319 tablet_bootstrap.cc:654] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.567797 13085 tablet_service.cc:1467] Processing CreateTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 (DEFAULT_TABLE table=test-table [id=45dd3f609fa745858e1a71dd64605f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.567934 13085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b4335984774af0b7b3fc5ce4132c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.568310 13344 ts_tablet_manager.cc:1939] T 09b1797e24704be7bf63ec60ed31d860 P db8c9698973143b4b7a1ad947d0149b0: Deleting consensus metadata
I20260928 17:56:52.568476 12792 catalog_manager.cc:4841] TS 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397): tablet 09b1797e24704be7bf63ec60ed31d860 (table test-table [id=02ec73e4334f49fa894383d097cfea68]) successfully deleted
I20260928 17:56:52.568667 12793 catalog_manager.cc:4841] TS db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989): tablet 09b1797e24704be7bf63ec60ed31d860 (table test-table [id=02ec73e4334f49fa894383d097cfea68]) successfully deleted
I20260928 17:56:52.568869 13319 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: No bootstrap required, opened a new log
I20260928 17:56:52.568907 13319 ts_tablet_manager.cc:1397] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:52.568989 13343 ts_tablet_manager.cc:1905] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:52.569039 13319 raft_consensus.cc:348] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.569085 13320 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Bootstrap starting.
I20260928 17:56:52.569118 13319 raft_consensus.cc:374] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.569151 13319 raft_consensus.cc:729] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.569229 13225 tablet_service.cc:1467] Processing CreateTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 (DEFAULT_TABLE table=test-table [id=45dd3f609fa745858e1a71dd64605f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:52.569227 13319 consensus_queue.cc:260] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.569334 13225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2b4335984774af0b7b3fc5ce4132c17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.569351 13319 ts_tablet_manager.cc:1428] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:52.569623 13320 tablet_bootstrap.cc:654] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.570772 13325 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Bootstrap starting.
I20260928 17:56:52.571184 13325 tablet_bootstrap.cc:654] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.571195 13343 ts_tablet_manager.cc:1918] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:52.571277 13343 log.cc:1196] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/09b1797e24704be7bf63ec60ed31d860
I20260928 17:56:52.571547 13343 ts_tablet_manager.cc:1939] T 09b1797e24704be7bf63ec60ed31d860 P 69971babcf6f44529935713c62644f2b: Deleting consensus metadata
I20260928 17:56:52.571911 13325 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: No bootstrap required, opened a new log
I20260928 17:56:52.571962 13325 ts_tablet_manager.cc:1397] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:52.571992 12793 catalog_manager.cc:4841] TS 69971babcf6f44529935713c62644f2b (127.5.46.67:43759): tablet 09b1797e24704be7bf63ec60ed31d860 (table test-table [id=02ec73e4334f49fa894383d097cfea68]) successfully deleted
I20260928 17:56:52.572108 13325 raft_consensus.cc:348] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.572177 13325 raft_consensus.cc:374] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.572197 13325 raft_consensus.cc:729] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69971babcf6f44529935713c62644f2b, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.572252 13325 consensus_queue.cc:260] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.572335 13325 ts_tablet_manager.cc:1428] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.576359 13320 tablet_bootstrap.cc:492] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: No bootstrap required, opened a new log
I20260928 17:56:52.576423 13320 ts_tablet_manager.cc:1397] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260928 17:56:52.576607 13320 raft_consensus.cc:348] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.576679 13320 raft_consensus.cc:374] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.576694 13320 raft_consensus.cc:729] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.576758 13320 consensus_queue.cc:260] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.576894 13320 ts_tablet_manager.cc:1428] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260928 17:56:52.608547 13292 tablet.cc:2369] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:56:52.883445 13323 raft_consensus.cc:482] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:52.883559 13323 raft_consensus.cc:504] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.883953 13323 leader_election.cc:288] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989), 69971babcf6f44529935713c62644f2b (127.5.46.67:43759)
I20260928 17:56:52.886880 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.884425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:51769 (local address 127.5.46.67:43759)
0928 17:56:52.884495 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.884497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.884548 (+ 51us) server_negotiation.cc:407] Connection header received
0928 17:56:52.884621 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.884622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.884632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.884683 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.884839 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.885290 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886183 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.886535 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886611 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.886620 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.886769 (+ 149us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.886806 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 17:56:52.886883 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.884234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:40815 (local address 127.5.46.66:33989)
0928 17:56:52.884346 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.884348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.884384 (+ 36us) server_negotiation.cc:407] Connection header received
0928 17:56:52.884473 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.884474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.884483 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.884547 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:52.884700 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.885178 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886144 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.886510 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886587 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.886598 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.886771 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.886809 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 17:56:52.886909 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.884354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.5.46.65:51769)
0928 17:56:52.884490 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.884501 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.884566 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.884695 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.884696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.884697 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.884769 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.884770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.885366 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.885367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.886097 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.886099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.886613 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.886696 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.886699 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.886763 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.886806 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260928 17:56:52.886914 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.884153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33989 (local address 127.5.46.65:40815)
0928 17:56:52.884305 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.884318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.884406 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.884551 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.884552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.884553 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:52.884635 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.884636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.885260 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.885261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.886055 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.886057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.886609 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.886611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.886688 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.886692 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.886763 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.886817 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260928 17:56:52.887187 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b4335984774af0b7b3fc5ce4132c17" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0" is_pre_election: true
I20260928 17:56:52.887223 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b4335984774af0b7b3fc5ce4132c17" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b" is_pre_election: true
I20260928 17:56:52.887315 13105 raft_consensus.cc:2454] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 0.
I20260928 17:56:52.887328 13245 raft_consensus.cc:2454] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 0.
I20260928 17:56:52.887485 12897 leader_election.cc:302] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, db8c9698973143b4b7a1ad947d0149b0; no voters:
I20260928 17:56:52.887641 13323 raft_consensus.cc:2781] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:52.887689 13323 raft_consensus.cc:482] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:52.887710 13323 raft_consensus.cc:3037] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.888473 13323 raft_consensus.cc:504] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.888607 13323 leader_election.cc:288] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 election: Requested vote from peers db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989), 69971babcf6f44529935713c62644f2b (127.5.46.67:43759)
I20260928 17:56:52.888803 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b4335984774af0b7b3fc5ce4132c17" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0"
I20260928 17:56:52.888877 13105 raft_consensus.cc:3037] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.888891 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2b4335984774af0b7b3fc5ce4132c17" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b"
I20260928 17:56:52.888976 13245 raft_consensus.cc:3037] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:52.889591 13245 raft_consensus.cc:2454] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 1.
I20260928 17:56:52.889601 13105 raft_consensus.cc:2454] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 1.
I20260928 17:56:52.889761 12897 leader_election.cc:302] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:52.889843 13323 raft_consensus.cc:2781] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:52.889997 13323 raft_consensus.cc:686] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 1 LEADER]: Becoming Leader. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Running, Role: LEADER
I20260928 17:56:52.890051 13323 consensus_queue.cc:237] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.890774 12808 catalog_manager.cc:5495] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23d9e96cb07145949cab38a565c1e080 (127.5.46.65). New cstate: current_term: 1 leader_uuid: "23d9e96cb07145949cab38a565c1e080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } health_report { overall_health: UNKNOWN } } }
Sep 28 17:56:52 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618211, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.126@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:52.952657 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.944738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.5.46.126:34833)
0928 17:56:52.944871 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:52.944873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:52.944964 (+ 91us) server_negotiation.cc:407] Connection header received
0928 17:56:52.945055 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:52.945056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:52.945068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:52.945163 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:52.945693 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.946215 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.946952 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.947141 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.947214 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.949660 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:52.949662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:52.949662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:52.949669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:52.951307 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.951765 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.951766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.951767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.951832 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:52.952122 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:52.952123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:52.952123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:52.952255 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:52.952324 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:52.952472 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 17:56:52.952577 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260928 17:56:52.952821 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:52.944648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:34833 (local address 127.0.0.1:46384)
0928 17:56:52.944876 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:52.944888 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:52.944984 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:52.945169 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:52.945170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:52.945558 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:52.945605 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.945608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.946315 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.946316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.946881 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:52.946883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:52.947212 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:52.947212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:52.947258 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:52.947882 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:52.947887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:52.949545 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:52.951415 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.951423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.951426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.951665 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.951967 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:52.951968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:52.951968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:52.952034 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:52.952323 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:52.952330 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:52.952377 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 17:56:52.952440 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 17:56:52.952650 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":90,"thread_start_us":126,"threads_started":1}
I20260928 17:56:52.954845 12808 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46384:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 17:56:52.955072 12808 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:52.956450 13085 tablet_service.cc:1467] Processing CreateTablet for tablet 73e3f98672be413ba22c99a45df5d6b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=632416f174f74260a394d2df69c57035]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:52.956473 12945 tablet_service.cc:1467] Processing CreateTablet for tablet 73e3f98672be413ba22c99a45df5d6b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=632416f174f74260a394d2df69c57035]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:52.956492 13225 tablet_service.cc:1467] Processing CreateTablet for tablet 73e3f98672be413ba22c99a45df5d6b3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=632416f174f74260a394d2df69c57035]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 17:56:52.956622 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e3f98672be413ba22c99a45df5d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.956622 13225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e3f98672be413ba22c99a45df5d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.956624 13085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73e3f98672be413ba22c99a45df5d6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:52.957720 13320 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: Bootstrap starting.
I20260928 17:56:52.958088 13325 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: Bootstrap starting.
I20260928 17:56:52.958101 13319 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: Bootstrap starting.
I20260928 17:56:52.958117 13320 tablet_bootstrap.cc:654] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.958515 13319 tablet_bootstrap.cc:654] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.958534 13325 tablet_bootstrap.cc:654] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:52.958760 13320 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: No bootstrap required, opened a new log
I20260928 17:56:52.958819 13320 ts_tablet_manager.cc:1397] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:52.958952 13320 raft_consensus.cc:348] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.959007 13320 raft_consensus.cc:374] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.959019 13320 raft_consensus.cc:729] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.959079 13320 consensus_queue.cc:260] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.959182 13320 ts_tablet_manager.cc:1428] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.959213 13325 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: No bootstrap required, opened a new log
I20260928 17:56:52.959252 13325 ts_tablet_manager.cc:1397] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:52.959223 13338 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:52.959350 13338 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:52.959419 13325 raft_consensus.cc:348] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.959491 13325 raft_consensus.cc:374] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.959518 13325 raft_consensus.cc:729] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69971babcf6f44529935713c62644f2b, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.959564 13325 consensus_queue.cc:260] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.959679 13325 ts_tablet_manager.cc:1428] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.959702 13319 tablet_bootstrap.cc:492] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: No bootstrap required, opened a new log
I20260928 17:56:52.959745 13319 ts_tablet_manager.cc:1397] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:52.959726 13327 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:52.959821 13327 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:52.959918 13319 raft_consensus.cc:348] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.959995 13319 raft_consensus.cc:374] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:52.960016 13319 raft_consensus.cc:729] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Initialized, Role: FOLLOWER
I20260928 17:56:52.960062 13319 consensus_queue.cc:260] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:52.960150 13319 ts_tablet_manager.cc:1428] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:52.960203 13323 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:52.960270 13323 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:53.252861 13327 raft_consensus.cc:482] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:53.252978 13327 raft_consensus.cc:504] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.253417 13327 leader_election.cc:288] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397), db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989)
I20260928 17:56:53.257905 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.253750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:56195 (local address 127.5.46.66:33989)
0928 17:56:53.253814 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.253816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.253942 (+ 126us) server_negotiation.cc:407] Connection header received
0928 17:56:53.253981 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.253982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.253999 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.254073 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:53.254342 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.255111 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.256906 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.257516 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.257582 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.257592 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.257786 (+ 194us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.257831 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 17:56:53.257910 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.253729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:38891 (local address 127.5.46.65:46397)
0928 17:56:53.253835 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.253837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.253942 (+ 105us) server_negotiation.cc:407] Connection header received
0928 17:56:53.253982 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.253982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.253995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.254073 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:56:53.254351 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.255127 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.256906 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.257479 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.257568 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.257582 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.257803 (+ 221us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.257846 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":49,"threads_started":1}
I20260928 17:56:53.257923 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.253654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46397 (local address 127.5.46.67:38891)
0928 17:56:53.253837 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.253849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.253908 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.254102 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.254103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.254107 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:53.254204 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.254206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.255243 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.255245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.256776 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.256781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.257596 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.257596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.257691 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.257700 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.257761 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.257818 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":133,"thread_start_us":56,"threads_started":1}
I20260928 17:56:53.257934 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.253654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33989 (local address 127.5.46.67:56195)
0928 17:56:53.253813 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.253827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.253908 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.254102 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.254103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.254106 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:56:53.254193 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.254196 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.255243 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.255245 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.256776 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.256781 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.257595 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.257596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.257691 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.257698 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.257761 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.257818 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":115,"thread_start_us":73,"threads_started":1}
I20260928 17:56:53.258370 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e3f98672be413ba22c99a45df5d6b3" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080" is_pre_election: true
I20260928 17:56:53.258495 12965 raft_consensus.cc:2454] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 0.
I20260928 17:56:53.258700 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e3f98672be413ba22c99a45df5d6b3" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0" is_pre_election: true
I20260928 17:56:53.258728 13176 leader_election.cc:302] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:53.258805 13105 raft_consensus.cc:2454] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 0.
I20260928 17:56:53.258909 13327 raft_consensus.cc:2781] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:53.258965 13327 raft_consensus.cc:482] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:53.258991 13327 raft_consensus.cc:3037] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.259795 13327 raft_consensus.cc:504] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.260007 13327 leader_election.cc:288] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 election: Requested vote from peers 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397), db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989)
I20260928 17:56:53.260181 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e3f98672be413ba22c99a45df5d6b3" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080"
I20260928 17:56:53.260190 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73e3f98672be413ba22c99a45df5d6b3" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0"
I20260928 17:56:53.260267 12965 raft_consensus.cc:3037] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.260282 13105 raft_consensus.cc:3037] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.260911 12965 raft_consensus.cc:2454] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 1.
I20260928 17:56:53.261020 13105 raft_consensus.cc:2454] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 1.
I20260928 17:56:53.261086 13176 leader_election.cc:302] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:53.261169 13327 raft_consensus.cc:2781] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:53.261294 13327 raft_consensus.cc:686] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [term 1 LEADER]: Becoming Leader. State: Replica: 69971babcf6f44529935713c62644f2b, State: Running, Role: LEADER
I20260928 17:56:53.261358 13327 consensus_queue.cc:237] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.261791 13365 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: TxnStatusTablet state changed. Reason: New leader 69971babcf6f44529935713c62644f2b. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.261932 13365 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:53.262126 13367 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 17:56:53.262250 13367 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 17:56:53.262187 12808 catalog_manager.cc:5495] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 69971babcf6f44529935713c62644f2b (127.5.46.67). New cstate: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:53.317618 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.314696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.5.46.67:43759)
0928 17:56:53.314761 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.314763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.314774 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:56:53.314857 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.314858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.314874 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.314937 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.315129 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.315633 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.316749 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.316941 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.317008 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.317272 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.317390 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.317452 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.317536 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.317554 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:56:53.317672 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.314609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.0.0.1:35956)
0928 17:56:53.314707 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.314722 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.314774 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.314959 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.314960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.314962 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:53.315033 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.315035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.315726 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.315728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.316641 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.316645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.317038 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.317039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.317123 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.317142 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.317464 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.317473 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.317531 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.317588 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 17:56:53.319283 12965 raft_consensus.cc:1261] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Refusing update from remote peer 69971babcf6f44529935713c62644f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.319362 13105 raft_consensus.cc:1261] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Refusing update from remote peer 69971babcf6f44529935713c62644f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.319597 13327 consensus_queue.cc:1035] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.319748 13365 consensus_queue.cc:1035] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.320379 13338 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: TxnStatusTablet state changed. Reason: New leader 69971babcf6f44529935713c62644f2b. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.320408 13323 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: TxnStatusTablet state changed. Reason: New leader 69971babcf6f44529935713c62644f2b. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.320488 13338 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:53.320499 13323 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:53.321475 13365 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.321584 13365 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:53.321556 13323 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.321630 13323 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 23d9e96cb07145949cab38a565c1e080: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:53.321707 13365 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.321704 13371 mvcc.cc:205] Tried to move back new op lower bound from 7334372201753075712 to 7334372201519304704. Current Snapshot: MvccSnapshot[applied={T|T < 7334372201753075712}]
I20260928 17:56:53.321779 13365 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P 69971babcf6f44529935713c62644f2b: This TxnStatusTablet replica's current role is: LEADER
I20260928 17:56:53.322118 13338 tablet_replica.cc:439] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } } }
I20260928 17:56:53.322209 13338 tablet_replica.cc:442] T 73e3f98672be413ba22c99a45df5d6b3 P db8c9698973143b4b7a1ad947d0149b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 17:56:53.326823 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.323768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.5.46.65:46397)
0928 17:56:53.323868 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.323871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.323968 (+ 97us) server_negotiation.cc:407] Connection header received
0928 17:56:53.324070 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.324071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.324102 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.324179 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.324401 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.324919 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.326017 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.326209 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.326291 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.326448 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.326544 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.326600 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.326668 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.326689 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 17:56:53.332530 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.329679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:58511 (local address 127.5.46.67:43759)
0928 17:56:53.329748 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.329751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.329904 (+ 153us) server_negotiation.cc:407] Connection header received
0928 17:56:53.329932 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.329933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.329944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.329996 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.330151 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.330828 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.331706 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.332023 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.332099 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.332232 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.332265 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.332321 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.332428 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.332453 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:53.332551 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.329598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.5.46.65:58511)
0928 17:56:53.329804 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.329814 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.329866 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.330027 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.330029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.330030 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:53.330072 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.330074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.330923 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.330924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.331608 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.331611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.332097 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.332097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.332139 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.332154 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.332326 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.332366 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.332406 (+ 40us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.332444 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":82,"threads_started":1}
I20260928 17:56:53.335361 13245 raft_consensus.cc:1261] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Refusing update from remote peer 23d9e96cb07145949cab38a565c1e080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.335352 13105 raft_consensus.cc:1261] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Refusing update from remote peer 23d9e96cb07145949cab38a565c1e080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.335609 13350 consensus_queue.cc:1035] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.335747 13323 consensus_queue.cc:1035] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.345026 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.342057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:43381 (local address 127.5.46.65:46397)
0928 17:56:53.342098 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.342100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.342226 (+ 126us) server_negotiation.cc:407] Connection header received
0928 17:56:53.342261 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.342262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.342272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.342341 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.342484 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.343218 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.344289 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.344486 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.344560 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.344740 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.344773 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.344828 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.344915 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.344938 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:56:53.345067 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.341992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46397 (local address 127.5.46.67:43381)
0928 17:56:53.342143 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.342155 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.342217 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.342344 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.342345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.342346 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:53.342419 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.342420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.343300 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.343301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.344163 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.344165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.344568 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.344570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.344646 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.344660 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.344858 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.344859 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.344911 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.344955 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260928 17:56:53.355702 12807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.355844 12807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.356498 12807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Sending DeleteTablet for 3 replicas of tablet f2b4335984774af0b7b3fc5ce4132c17
I20260928 17:56:53.356784 12945 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:56714
I20260928 17:56:53.357057 13085 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41618
I20260928 17:56:53.357106 13225 tablet_service.cc:1514] Processing DeleteTablet for tablet f2b4335984774af0b7b3fc5ce4132c17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35948
I20260928 17:56:53.357465 13391 tablet_replica.cc:330] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: stopping tablet replica
I20260928 17:56:53.357553 13391 raft_consensus.cc:2229] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:53.357650 13391 raft_consensus.cc:2258] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.357770 13393 tablet_replica.cc:330] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: stopping tablet replica
I20260928 17:56:53.357844 13393 raft_consensus.cc:2229] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.357890 13393 raft_consensus.cc:2258] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.358017 13392 tablet_replica.cc:330] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: stopping tablet replica
I20260928 17:56:53.358083 13392 raft_consensus.cc:2229] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.358130 13392 raft_consensus.cc:2258] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.358345 13393 ts_tablet_manager.cc:1905] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.360303 13393 ts_tablet_manager.cc:1918] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:53.360368 13393 log.cc:1196] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/f2b4335984774af0b7b3fc5ce4132c17
I20260928 17:56:53.360639 13393 ts_tablet_manager.cc:1939] T f2b4335984774af0b7b3fc5ce4132c17 P 69971babcf6f44529935713c62644f2b: Deleting consensus metadata
I20260928 17:56:53.360782 13391 ts_tablet_manager.cc:1905] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.362483 13391 ts_tablet_manager.cc:1918] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:53.362548 13391 log.cc:1196] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/f2b4335984774af0b7b3fc5ce4132c17
I20260928 17:56:53.362664 13392 ts_tablet_manager.cc:1905] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.362658 12793 catalog_manager.cc:4841] TS 69971babcf6f44529935713c62644f2b (127.5.46.67:43759): tablet f2b4335984774af0b7b3fc5ce4132c17 (table test-table [id=45dd3f609fa745858e1a71dd64605f53]) successfully deleted
I20260928 17:56:53.362780 13391 ts_tablet_manager.cc:1939] T f2b4335984774af0b7b3fc5ce4132c17 P 23d9e96cb07145949cab38a565c1e080: Deleting consensus metadata
I20260928 17:56:53.364173 12792 catalog_manager.cc:4841] TS 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397): tablet f2b4335984774af0b7b3fc5ce4132c17 (table test-table [id=45dd3f609fa745858e1a71dd64605f53]) successfully deleted
I20260928 17:56:53.365309 13392 ts_tablet_manager.cc:1918] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:53.365374 13392 log.cc:1196] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/f2b4335984774af0b7b3fc5ce4132c17
I20260928 17:56:53.365617 13392 ts_tablet_manager.cc:1939] T f2b4335984774af0b7b3fc5ce4132c17 P db8c9698973143b4b7a1ad947d0149b0: Deleting consensus metadata
I20260928 17:56:53.365942 12793 catalog_manager.cc:4841] TS db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989): tablet f2b4335984774af0b7b3fc5ce4132c17 (table test-table [id=45dd3f609fa745858e1a71dd64605f53]) successfully deleted
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[12763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 17:56:53.388044 13358 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:53.382001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.5.46.126:34833)
0928 17:56:53.382067 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.382070 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.382238 (+ 168us) server_negotiation.cc:407] Connection header received
0928 17:56:53.382323 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.382324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.382337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.382396 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:53.382983 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.383495 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.384362 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.384575 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.384670 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.387988 (+ 3318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46392: BlockingRecv error: recv got EOF from 127.0.0.1:46392 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[12763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM
I20260928 17:56:53.397516 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.389157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46404 (local address 127.5.46.126:34833)
0928 17:56:53.389203 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.389206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.389487 (+ 281us) server_negotiation.cc:407] Connection header received
0928 17:56:53.389560 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.389560 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.389575 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.389638 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:53.390164 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.390709 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.391521 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.391745 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.391841 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.394502 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:53.394504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:53.394504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:53.394516 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:53.396210 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:53.396613 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:53.396614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:53.396615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:53.396680 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:53.396901 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:53.396902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:53.396903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:53.397032 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:53.397155 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.397267 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.397391 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:56:53.399349 12807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:53.399544 12807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:53.401150 12945 tablet_service.cc:1467] Processing CreateTablet for tablet eb3a65032f594c64af975c4643f5ed52 (DEFAULT_TABLE table=test-table [id=cd413c819c98422aaac1562ae5dcd2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.401247 13085 tablet_service.cc:1467] Processing CreateTablet for tablet eb3a65032f594c64af975c4643f5ed52 (DEFAULT_TABLE table=test-table [id=cd413c819c98422aaac1562ae5dcd2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.401298 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3a65032f594c64af975c4643f5ed52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.401365 13085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3a65032f594c64af975c4643f5ed52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.402660 13319 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Bootstrap starting.
I20260928 17:56:53.402693 13225 tablet_service.cc:1467] Processing CreateTablet for tablet eb3a65032f594c64af975c4643f5ed52 (DEFAULT_TABLE table=test-table [id=cd413c819c98422aaac1562ae5dcd2b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.402823 13225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb3a65032f594c64af975c4643f5ed52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.403132 13319 tablet_bootstrap.cc:654] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.404328 13319 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: No bootstrap required, opened a new log
I20260928 17:56:53.404374 13319 ts_tablet_manager.cc:1397] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:53.404459 13320 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Bootstrap starting.
I20260928 17:56:53.404533 13319 raft_consensus.cc:348] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.404606 13319 raft_consensus.cc:374] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.404624 13319 raft_consensus.cc:729] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.404666 13319 consensus_queue.cc:260] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.404769 13320 tablet_bootstrap.cc:654] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.404788 13319 ts_tablet_manager.cc:1428] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:53.405068 13325 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Bootstrap starting.
I20260928 17:56:53.405465 13325 tablet_bootstrap.cc:654] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.406380 13320 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: No bootstrap required, opened a new log
I20260928 17:56:53.406453 13320 ts_tablet_manager.cc:1397] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:56:53.406597 13320 raft_consensus.cc:348] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.406679 13320 raft_consensus.cc:374] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.406708 13320 raft_consensus.cc:729] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.406750 13320 consensus_queue.cc:260] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.406845 13320 ts_tablet_manager.cc:1428] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:53.407378 13325 tablet_bootstrap.cc:492] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: No bootstrap required, opened a new log
I20260928 17:56:53.407430 13325 ts_tablet_manager.cc:1397] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 17:56:53.407593 13325 raft_consensus.cc:348] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.407670 13325 raft_consensus.cc:374] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.407697 13325 raft_consensus.cc:729] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69971babcf6f44529935713c62644f2b, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.407742 13325 consensus_queue.cc:260] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.407832 13325 ts_tablet_manager.cc:1428] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:53.440286 13389 raft_consensus.cc:482] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:53.440400 13389 raft_consensus.cc:504] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.440587 13389 leader_election.cc:288] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989), 69971babcf6f44529935713c62644f2b (127.5.46.67:43759)
I20260928 17:56:53.440824 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3a65032f594c64af975c4643f5ed52" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b" is_pre_election: true
I20260928 17:56:53.440820 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3a65032f594c64af975c4643f5ed52" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0" is_pre_election: true
I20260928 17:56:53.440927 13245 raft_consensus.cc:2454] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 0.
I20260928 17:56:53.440940 13105 raft_consensus.cc:2454] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 0.
I20260928 17:56:53.441084 12897 leader_election.cc:302] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:53.441182 13389 raft_consensus.cc:2781] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:53.441304 13389 raft_consensus.cc:482] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:53.441330 13389 raft_consensus.cc:3037] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.442076 13389 raft_consensus.cc:504] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.442222 13389 leader_election.cc:288] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 election: Requested vote from peers db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989), 69971babcf6f44529935713c62644f2b (127.5.46.67:43759)
I20260928 17:56:53.442381 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3a65032f594c64af975c4643f5ed52" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69971babcf6f44529935713c62644f2b"
I20260928 17:56:53.442382 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb3a65032f594c64af975c4643f5ed52" candidate_uuid: "23d9e96cb07145949cab38a565c1e080" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0"
I20260928 17:56:53.442474 13105 raft_consensus.cc:3037] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.442476 13245 raft_consensus.cc:3037] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.443120 13245 raft_consensus.cc:2454] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 1.
I20260928 17:56:53.443274 13105 raft_consensus.cc:2454] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23d9e96cb07145949cab38a565c1e080 in term 1.
I20260928 17:56:53.443287 12897 leader_election.cc:302] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:53.443416 13389 raft_consensus.cc:2781] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:53.443452 13389 raft_consensus.cc:686] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 1 LEADER]: Becoming Leader. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Running, Role: LEADER
I20260928 17:56:53.443499 13389 consensus_queue.cc:237] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.444151 12806 catalog_manager.cc:5495] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23d9e96cb07145949cab38a565c1e080 (127.5.46.65). New cstate: current_term: 1 leader_uuid: "23d9e96cb07145949cab38a565c1e080" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } health_report { overall_health: UNKNOWN } } }
I20260928 17:56:53.452600 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.450350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.5.46.65:46397)
0928 17:56:53.450401 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.450403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.450412 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:53.450439 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.450440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.450449 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.450504 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.450626 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.451099 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.451910 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.452113 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.452188 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.452357 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.452392 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.452441 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.452498 (+ 57us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.452515 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 17:56:53.453356 13245 raft_consensus.cc:1261] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Refusing update from remote peer 23d9e96cb07145949cab38a565c1e080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.453389 13105 raft_consensus.cc:1261] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Refusing update from remote peer 23d9e96cb07145949cab38a565c1e080: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.453601 13389 consensus_queue.cc:1035] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.453753 13350 consensus_queue.cc:1035] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.456599 12806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.456709 12806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.457304 12806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Sending DeleteTablet for 3 replicas of tablet eb3a65032f594c64af975c4643f5ed52
I20260928 17:56:53.457559 13225 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3a65032f594c64af975c4643f5ed52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35948
I20260928 17:56:53.457563 13085 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3a65032f594c64af975c4643f5ed52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41618
I20260928 17:56:53.457653 13392 tablet_replica.cc:330] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: stopping tablet replica
I20260928 17:56:53.457661 12945 tablet_service.cc:1514] Processing DeleteTablet for tablet eb3a65032f594c64af975c4643f5ed52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:56714
I20260928 17:56:53.457712 13392 raft_consensus.cc:2229] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.457738 13393 tablet_replica.cc:330] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: stopping tablet replica
I20260928 17:56:53.457757 13392 raft_consensus.cc:2258] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.457792 13393 raft_consensus.cc:2229] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.457845 13393 raft_consensus.cc:2258] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.457922 13391 tablet_replica.cc:330] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: stopping tablet replica
I20260928 17:56:53.457983 13391 raft_consensus.cc:2229] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:53.458055 13391 raft_consensus.cc:2258] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.458141 13392 ts_tablet_manager.cc:1905] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.458448 13391 ts_tablet_manager.cc:1905] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.458451 12806 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:56:53.458590 12806 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:56:53.460134 13391 ts_tablet_manager.cc:1918] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:53.460208 13391 log.cc:1196] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/eb3a65032f594c64af975c4643f5ed52
I20260928 17:56:53.460270 13085 tablet_service.cc:1467] Processing CreateTablet for tablet 58b186674a134983aec278888143c713 (DEFAULT_TABLE table=test-table [id=dba61e2dd9644855bbc0f2272b4abd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.460405 13225 tablet_service.cc:1467] Processing CreateTablet for tablet 58b186674a134983aec278888143c713 (DEFAULT_TABLE table=test-table [id=dba61e2dd9644855bbc0f2272b4abd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.460465 13391 ts_tablet_manager.cc:1939] T eb3a65032f594c64af975c4643f5ed52 P 23d9e96cb07145949cab38a565c1e080: Deleting consensus metadata
I20260928 17:56:53.460541 13225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b186674a134983aec278888143c713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.460659 13392 ts_tablet_manager.cc:1918] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:53.460711 13392 log.cc:1196] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/eb3a65032f594c64af975c4643f5ed52
I20260928 17:56:53.460906 13392 ts_tablet_manager.cc:1939] T eb3a65032f594c64af975c4643f5ed52 P db8c9698973143b4b7a1ad947d0149b0: Deleting consensus metadata
I20260928 17:56:53.461210 13393 ts_tablet_manager.cc:1905] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.461745 13325 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Bootstrap starting.
I20260928 17:56:53.461930 13085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b186674a134983aec278888143c713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.462100 13325 tablet_bootstrap.cc:654] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.462775 13393 ts_tablet_manager.cc:1918] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:56:53.462838 13393 log.cc:1196] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/eb3a65032f594c64af975c4643f5ed52
I20260928 17:56:53.463063 13393 ts_tablet_manager.cc:1939] T eb3a65032f594c64af975c4643f5ed52 P 69971babcf6f44529935713c62644f2b: Deleting consensus metadata
I20260928 17:56:53.463313 13320 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Bootstrap starting.
I20260928 17:56:53.463596 13320 tablet_bootstrap.cc:654] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.463956 12793 catalog_manager.cc:4841] TS 69971babcf6f44529935713c62644f2b (127.5.46.67:43759): tablet eb3a65032f594c64af975c4643f5ed52 (table test-table [id=cd413c819c98422aaac1562ae5dcd2b3]) successfully deleted
I20260928 17:56:53.464004 12792 catalog_manager.cc:4841] TS 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397): tablet eb3a65032f594c64af975c4643f5ed52 (table test-table [id=cd413c819c98422aaac1562ae5dcd2b3]) successfully deleted
I20260928 17:56:53.464190 13320 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: No bootstrap required, opened a new log
I20260928 17:56:53.464257 13320 ts_tablet_manager.cc:1397] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:53.464371 13320 raft_consensus.cc:348] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.464443 13320 raft_consensus.cc:374] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.464468 13320 raft_consensus.cc:729] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8c9698973143b4b7a1ad947d0149b0, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.464445 12945 tablet_service.cc:1467] Processing CreateTablet for tablet 58b186674a134983aec278888143c713 (DEFAULT_TABLE table=test-table [id=dba61e2dd9644855bbc0f2272b4abd2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:56:53.464509 13320 consensus_queue.cc:260] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.464546 12945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58b186674a134983aec278888143c713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:53.464603 13320 ts_tablet_manager.cc:1428] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:53.464924 12793 catalog_manager.cc:4841] TS db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989): tablet eb3a65032f594c64af975c4643f5ed52 (table test-table [id=cd413c819c98422aaac1562ae5dcd2b3]) successfully deleted
I20260928 17:56:53.465694 13319 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: Bootstrap starting.
I20260928 17:56:53.465926 13325 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: No bootstrap required, opened a new log
I20260928 17:56:53.465961 13325 ts_tablet_manager.cc:1397] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260928 17:56:53.466074 13325 raft_consensus.cc:348] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.466101 13319 tablet_bootstrap.cc:654] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:53.466120 13325 raft_consensus.cc:374] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.466135 13325 raft_consensus.cc:729] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69971babcf6f44529935713c62644f2b, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.466198 13325 consensus_queue.cc:260] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.466284 13325 ts_tablet_manager.cc:1428] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:53.466815 13319 tablet_bootstrap.cc:492] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: No bootstrap required, opened a new log
I20260928 17:56:53.466866 13319 ts_tablet_manager.cc:1397] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:53.467006 13319 raft_consensus.cc:348] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.467068 13319 raft_consensus.cc:374] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:53.467087 13319 raft_consensus.cc:729] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d9e96cb07145949cab38a565c1e080, State: Initialized, Role: FOLLOWER
I20260928 17:56:53.467134 13319 consensus_queue.cc:260] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.467221 13319 ts_tablet_manager.cc:1428] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:53.604801 13385 raft_consensus.cc:482] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:56:53.604971 13385 raft_consensus.cc:504] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.605216 13385 leader_election.cc:288] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397), db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989)
I20260928 17:56:53.605490 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b186674a134983aec278888143c713" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080" is_pre_election: true
I20260928 17:56:53.605602 12965 raft_consensus.cc:2454] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 0.
I20260928 17:56:53.605551 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b186674a134983aec278888143c713" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0" is_pre_election: true
I20260928 17:56:53.605652 13105 raft_consensus.cc:2454] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 0.
I20260928 17:56:53.605762 13176 leader_election.cc:302] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23d9e96cb07145949cab38a565c1e080, 69971babcf6f44529935713c62644f2b; no voters:
I20260928 17:56:53.605863 13385 raft_consensus.cc:2781] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:56:53.605921 13385 raft_consensus.cc:482] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:56:53.605957 13385 raft_consensus.cc:3037] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.606748 13385 raft_consensus.cc:504] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.606910 13385 leader_election.cc:288] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 election: Requested vote from peers 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397), db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989)
I20260928 17:56:53.607082 13105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b186674a134983aec278888143c713" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db8c9698973143b4b7a1ad947d0149b0"
I20260928 17:56:53.607077 12965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58b186674a134983aec278888143c713" candidate_uuid: "69971babcf6f44529935713c62644f2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d9e96cb07145949cab38a565c1e080"
I20260928 17:56:53.607160 13105 raft_consensus.cc:3037] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.607188 12965 raft_consensus.cc:3037] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:53.608012 12965 raft_consensus.cc:2454] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 1.
I20260928 17:56:53.608021 13105 raft_consensus.cc:2454] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69971babcf6f44529935713c62644f2b in term 1.
I20260928 17:56:53.608251 13177 leader_election.cc:302] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69971babcf6f44529935713c62644f2b, db8c9698973143b4b7a1ad947d0149b0; no voters:
I20260928 17:56:53.608413 13385 raft_consensus.cc:2781] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:53.608469 13385 raft_consensus.cc:686] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 1 LEADER]: Becoming Leader. State: Replica: 69971babcf6f44529935713c62644f2b, State: Running, Role: LEADER
I20260928 17:56:53.608529 13385 consensus_queue.cc:237] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } }
I20260928 17:56:53.609184 12806 catalog_manager.cc:5495] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b reported cstate change: term changed from 0 to 1, leader changed from <none> to 69971babcf6f44529935713c62644f2b (127.5.46.67). New cstate: current_term: 1 leader_uuid: "69971babcf6f44529935713c62644f2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69971babcf6f44529935713c62644f2b" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 43759 } health_report { overall_health: HEALTHY } } }
I20260928 17:56:53.616969 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.614585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.5.46.67:43759)
0928 17:56:53.614657 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.614659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.614669 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:56:53.614691 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.614692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.614703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.614794 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.614947 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.615411 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.616269 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.616470 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.616551 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.616735 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.616763 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.616819 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.616888 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.616908 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 17:56:53.621563 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.619150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:43759 (local address 127.5.46.67:43303)
0928 17:56:53.619207 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.619250 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:53.619295 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:53.619446 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:53.619447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:53.619448 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 17:56:53.619501 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.619502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.620194 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.620195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.620761 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:53.620763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.621160 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.621161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:53.621211 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.621217 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.621397 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.621398 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 17:56:53.621450 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:56:53.621498 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":3}
I20260928 17:56:53.621563 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.619193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:43303 (local address 127.5.46.67:43759)
0928 17:56:53.619274 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:53.619276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:53.619284 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:53.619381 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:53.619382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:53.619390 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:53.619444 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:56:53.619562 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.620111 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.620871 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:53.621075 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:53.621160 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:53.621293 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:56:53.621336 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:56:53.621398 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:53.621454 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 17:56:53.621475 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":62}
I20260928 17:56:53.623669 12965 raft_consensus.cc:1261] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Refusing update from remote peer 69971babcf6f44529935713c62644f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.623708 13104 raft_consensus.cc:1261] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Refusing update from remote peer 69971babcf6f44529935713c62644f2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:56:53.623930 13365 consensus_queue.cc:1035] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d9e96cb07145949cab38a565c1e080" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 46397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.624017 13327 consensus_queue.cc:1035] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db8c9698973143b4b7a1ad947d0149b0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:56:53.634658 12806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.634796 12806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:56:53.635432 12806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99b3c955a6b945de99a30ed5f0e611fa: Sending DeleteTablet for 3 replicas of tablet 58b186674a134983aec278888143c713
I20260928 17:56:53.635663 12945 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b186674a134983aec278888143c713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:56714
I20260928 17:56:53.635707 13085 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b186674a134983aec278888143c713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:41618
I20260928 17:56:53.635778 13391 tablet_replica.cc:330] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: stopping tablet replica
I20260928 17:56:53.635815 13392 tablet_replica.cc:330] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: stopping tablet replica
I20260928 17:56:53.635828 13391 raft_consensus.cc:2229] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.635804 13225 tablet_service.cc:1514] Processing DeleteTablet for tablet 58b186674a134983aec278888143c713 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:56:53 UTC) from {username='oryx', principal='oryx/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35948
I20260928 17:56:53.635891 13392 raft_consensus.cc:2229] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:56:53.635903 13391 raft_consensus.cc:2258] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.635938 13392 raft_consensus.cc:2258] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.635933 13393 tablet_replica.cc:330] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: stopping tablet replica
I20260928 17:56:53.635984 13393 raft_consensus.cc:2229] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:56:53.636083 13393 raft_consensus.cc:2258] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:56:53.636479 13392 ts_tablet_manager.cc:1905] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.636479 13393 ts_tablet_manager.cc:1905] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.636523 13391 ts_tablet_manager.cc:1905] T 58b186674a134983aec278888143c713 P 23d9e96cb07145949cab38a565c1e080: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:56:53.638293 13392 ts_tablet_manager.cc:1918] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:53.638358 13392 log.cc:1196] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/58b186674a134983aec278888143c713
I20260928 17:56:53.638391 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12880
I20260928 17:56:53.638578 13392 ts_tablet_manager.cc:1939] T 58b186674a134983aec278888143c713 P db8c9698973143b4b7a1ad947d0149b0: Deleting consensus metadata
I20260928 17:56:53.640442 12793 catalog_manager.cc:4841] TS db8c9698973143b4b7a1ad947d0149b0 (127.5.46.66:33989): tablet 58b186674a134983aec278888143c713 (table test-table [id=dba61e2dd9644855bbc0f2272b4abd2e]) successfully deleted
I20260928 17:56:53.644070 13393 ts_tablet_manager.cc:1918] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 17:56:53.644160 13393 log.cc:1196] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/58b186674a134983aec278888143c713
I20260928 17:56:53.644431 13393 ts_tablet_manager.cc:1939] T 58b186674a134983aec278888143c713 P 69971babcf6f44529935713c62644f2b: Deleting consensus metadata
I20260928 17:56:53.644873 12793 catalog_manager.cc:4841] TS 69971babcf6f44529935713c62644f2b (127.5.46.67:43759): tablet 58b186674a134983aec278888143c713 (table test-table [id=dba61e2dd9644855bbc0f2272b4abd2e]) successfully deleted
W20260928 17:56:53.645757 12792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:46397 (error 108)
I20260928 17:56:53.645849 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13020
I20260928 17:56:53.647511 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:53.647260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:46397 (local address 127.0.0.1:56750)
0928 17:56:53.647402 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:53.647455 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:46397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
W20260928 17:56:53.647637 12792 catalog_manager.cc:4584] TS 23d9e96cb07145949cab38a565c1e080 (127.5.46.65:46397): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58b186674a134983aec278888143c713: Network error: Client connection negotiation failed: client connection to 127.5.46.65:46397: connect: Connection refused (error 111)
I20260928 17:56:53.654066 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13160
I20260928 17:56:53.659214 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 12776
2026-09-28T17:56:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3138 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[13425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[13425](info): set up 2 sockets
Sep 28 17:56:53 dist-test-slave-z2t3 krb5kdc[13425](info): commencing operation
krb5kdc: starting...
W20260928 17:56:54.712670 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:54 dist-test-slave-z2t3 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:54.840022 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41103
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:41103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:54.926440 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:54.926666 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:54.926689 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:54.927824 13438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:54.927896 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:54.927911 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:54.927924 13438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:54.927937 13438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:54.929446 13438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:41103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:54.929654 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:54.929845 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:54.934731 13444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:54.934722 13446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:54.935035 13438 server_base.cc:1034] running on GCE node
W20260928 17:56:54.934725 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:54.935325 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:54.935577 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:54.936724 13438 hybrid_clock.cc:648] HybridClock initialized: now 1790618214936702 us; error 36 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T17:56:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1275 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:56:54 dist-test-slave-z2t3 krb5kdc[13451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:56:54 dist-test-slave-z2t3 krb5kdc[13451](info): set up 2 sockets
Sep 28 17:56:54 dist-test-slave-z2t3 krb5kdc[13451](info): commencing operation
krb5kdc: starting...
W20260928 17:56:56.016601 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:56:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:56:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:56:56.144263 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46433
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:56.229692 13464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:56.229918 13464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:56.229943 13464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:56.231161 13464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:56.231215 13464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:56.231230 13464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:56.231287 13464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:56.231307 13464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:56.232925 13464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:56.233201 13464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:56.233498 13464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:56.238643 13472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.238643 13470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.238675 13469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.238860 13464 server_base.cc:1034] running on GCE node
I20260928 17:56:56.239303 13464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:56.239620 13464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:56.240785 13464 hybrid_clock.cc:648] HybridClock initialized: now 1790618216240772 us; error 34 us; skew 500 ppm
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:56.243392 13464 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:56.243968 13464 webserver.cc:460] Webserver started at http://127.5.46.126:41801/ using document root <none> and password file <none>
I20260928 17:56:56.244215 13464 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:56.244284 13464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:56.244433 13464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:56.245523 13464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "b00d4f7de11f46f6aefdd6affc4420e2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "7c3c988b3bbad7365bc2a980a0d05dd9"
server_key_iv: "de353395e88a1e6a1aa35cf9e0d165db"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.245894 13464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "b00d4f7de11f46f6aefdd6affc4420e2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "7c3c988b3bbad7365bc2a980a0d05dd9"
server_key_iv: "de353395e88a1e6a1aa35cf9e0d165db"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.247251 13464 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:56.248070 13479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.248260 13464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:56.248369 13464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "b00d4f7de11f46f6aefdd6affc4420e2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "7c3c988b3bbad7365bc2a980a0d05dd9"
server_key_iv: "de353395e88a1e6a1aa35cf9e0d165db"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.248445 13464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:56.265946 13464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:56.268115 13464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:56.268280 13464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:56.272253 13464 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46433
I20260928 17:56:56.272657 13464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:56.272631 13531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46433 every 8 connection(s)
I20260928 17:56:56.273386 13532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:56:56.275688 13532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Bootstrap starting.
I20260928 17:56:56.276315 13532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Neither blocks nor log segments found. Creating new log.
I20260928 17:56:56.276552 13532 log.cc:824] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:56.277189 13532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: No bootstrap required, opened a new log
I20260928 17:56:56.278896 13532 raft_consensus.cc:348] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:56.279044 13532 raft_consensus.cc:374] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:56:56.279066 13532 raft_consensus.cc:729] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00d4f7de11f46f6aefdd6affc4420e2, State: Initialized, Role: FOLLOWER
I20260928 17:56:56.279210 13532 consensus_queue.cc:260] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:56.279294 13532 raft_consensus.cc:388] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:56.279318 13532 raft_consensus.cc:482] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:56.279346 13532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:56:56.279793 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13464
I20260928 17:56:56.279937 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:56:56.279950 13532 raft_consensus.cc:504] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:56.280082 13532 leader_election.cc:302] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b00d4f7de11f46f6aefdd6affc4420e2; no voters:
I20260928 17:56:56.280192 5305 external_mini_cluster.cc:1428] Setting key 5616b2a11190fd1c71e883aa8afa77f3
I20260928 17:56:56.280277 13532 leader_election.cc:288] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:56:56.280555 13532 sys_catalog.cc:564] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:56.280323 13535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:56:56.280918 13535 raft_consensus.cc:686] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 LEADER]: Becoming Leader. State: Replica: b00d4f7de11f46f6aefdd6affc4420e2, State: Running, Role: LEADER
I20260928 17:56:56.281109 13535 consensus_queue.cc:237] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:56.281538 13538 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b00d4f7de11f46f6aefdd6affc4420e2. Latest consensus state: current_term: 1 leader_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } } }
I20260928 17:56:56.281646 13538 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:56.282267 13536 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } } }
I20260928 17:56:56.282384 13549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:56.282776 13536 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:56.283177 13549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:56.285379 13549 catalog_manager.cc:1347] Generated new cluster ID: 54df42caa2584f34b1da1f64a208d5df
I20260928 17:56:56.285444 13549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:56.293015 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.281021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52602 (local address 127.5.46.126:46433)
0928 17:56:56.281524 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:56.281534 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:56.281569 (+ 35us) server_negotiation.cc:407] Connection header received
0928 17:56:56.281691 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:56.281700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:56.281760 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:56.281949 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:56.282679 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.284078 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.284894 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.285525 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.285641 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.288564 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:56.288568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:56.288569 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:56.288582 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:56.291261 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.291696 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.291699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.291700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.291812 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.292065 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.292065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.292066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.292374 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:56.292511 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:56.292651 (+ 140us) server_negotiation.cc:299] Negotiation successful
0928 17:56:56.292784 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":81,"threads_started":1}
I20260928 17:56:56.300191 13549 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:56:56.300792 13549 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:56.305063 13549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Generated new TSK 0
I20260928 17:56:56.305274 13549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:56:56.343675 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:56.435257 13558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:56.435477 13558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:56.435501 13558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:56.436744 13558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:56.436797 13558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:56.436825 13558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:56.438383 13558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:56.438645 13558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:56.438895 13558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:56.444061 13563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.444060 13564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.444044 13566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.444262 13558 server_base.cc:1034] running on GCE node
I20260928 17:56:56.444512 13558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:56.444764 13558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:56.445931 13558 hybrid_clock.cc:648] HybridClock initialized: now 1790618216445920 us; error 27 us; skew 500 ppm
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:56.448467 13558 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:56:56.449005 13558 webserver.cc:460] Webserver started at http://127.5.46.65:33277/ using document root <none> and password file <none>
I20260928 17:56:56.449246 13558 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:56.449314 13558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:56.449455 13558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:56.450604 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "3fbe388e22a14cfa87b9b42018a49f14"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "386adc76feb35a27da7c5b1d69d62f32"
server_key_iv: "39af13aa63fa48d23ff070c36d798a55"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.450994 13558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "3fbe388e22a14cfa87b9b42018a49f14"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "386adc76feb35a27da7c5b1d69d62f32"
server_key_iv: "39af13aa63fa48d23ff070c36d798a55"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.452570 13558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:56:56.453347 13573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.453536 13558 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:56.453639 13558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "3fbe388e22a14cfa87b9b42018a49f14"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "386adc76feb35a27da7c5b1d69d62f32"
server_key_iv: "39af13aa63fa48d23ff070c36d798a55"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.453713 13558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:56.477540 13558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:56.479805 13558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:56.480010 13558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:56.480252 13558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:56.480587 13558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:56.480646 13558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.480690 13558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:56.480717 13558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.486799 13558 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:39705
I20260928 17:56:56.486842 13686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:39705 every 8 connection(s)
I20260928 17:56:56.487205 13558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:56.489624 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13558
I20260928 17:56:56.489734 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:56:56.489917 5305 external_mini_cluster.cc:1428] Setting key 1240f65cd499700df056713743fc0518
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:56.497823 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.488102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:57441 (local address 127.5.46.126:46433)
0928 17:56:56.488148 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:56.488150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:56.488391 (+ 241us) server_negotiation.cc:407] Connection header received
0928 17:56:56.488937 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:56.488939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:56.488953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:56.489011 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:56.490238 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.490890 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.491938 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.492161 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.492243 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.494925 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:56.494926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:56.494926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:56.494934 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:56.496372 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.496883 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.496884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.496884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.496931 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.497205 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.497205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.497206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.497304 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:56.497354 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:56.497694 (+ 340us) server_negotiation.cc:299] Negotiation successful
0928 17:56:56.497758 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:56:56.498153 13689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.488075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:57441)
0928 17:56:56.488262 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:56.488300 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:56.488807 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:56.489093 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:56.489097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:56.489845 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:56.490140 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.490146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.490989 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.490990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.491795 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.491797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.492251 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.492252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.492398 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.493102 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:56.493111 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:56.494829 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:56.496456 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.496464 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.496466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.496787 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.496996 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.496997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.496998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.497135 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.497373 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:56.497379 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:56.497550 (+ 171us) client_negotiation.cc:769] Sending connection context
0928 17:56:56.497683 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 17:56:56.497873 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
I20260928 17:56:56.498729 13687 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46433
I20260928 17:56:56.498832 13687 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:56.499032 13687 heartbeater.cc:511] Master 127.5.46.126:46433 requested a full tablet report, sending...
I20260928 17:56:56.499564 13496 ts_manager.cc:194] Registered new tserver with Master: 3fbe388e22a14cfa87b9b42018a49f14 (127.5.46.65:39705)
I20260928 17:56:56.500197 13496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:57441
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:56:56.515718 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:56.599459 13692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:56.599732 13692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:56.599773 13692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:56.601121 13692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:56.601199 13692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:56.601245 13692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:56.602798 13692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:56.603111 13692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:56.603374 13692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:56.608723 13697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.608726 13700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.608817 13692 server_base.cc:1034] running on GCE node
W20260928 17:56:56.608726 13698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.609184 13692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:56.609475 13692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:56.610646 13692 hybrid_clock.cc:648] HybridClock initialized: now 1790618216610641 us; error 34 us; skew 500 ppm
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:56.613183 13692 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:56:56.613710 13692 webserver.cc:460] Webserver started at http://127.5.46.66:33945/ using document root <none> and password file <none>
I20260928 17:56:56.613956 13692 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:56.614029 13692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:56.614187 13692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:56.615358 13692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "44ce5d0c3ee349c9b30fb29ede33b420"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "29bbd35cd27a427eef2809ea07570c1f"
server_key_iv: "194d6f6d87f31b5a084fb9531063a7e5"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.615731 13692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "44ce5d0c3ee349c9b30fb29ede33b420"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "29bbd35cd27a427eef2809ea07570c1f"
server_key_iv: "194d6f6d87f31b5a084fb9531063a7e5"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.617141 13692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:56.618047 13707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.618255 13692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:56:56.618405 13692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "44ce5d0c3ee349c9b30fb29ede33b420"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "29bbd35cd27a427eef2809ea07570c1f"
server_key_iv: "194d6f6d87f31b5a084fb9531063a7e5"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.618494 13692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:56.629877 13692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:56.632000 13692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:56.632189 13692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:56.632455 13692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:56.632791 13692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:56.632853 13692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.632895 13692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:56.632937 13692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.638993 13692 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:42853
I20260928 17:56:56.639034 13820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:42853 every 8 connection(s)
I20260928 17:56:56.639364 13692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:56:56.640972 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13692
I20260928 17:56:56.641105 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:56:56.641276 5305 external_mini_cluster.cc:1428] Setting key 0391f976f8506854c50223c02d7d2635
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:56.649919 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.640146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:33617 (local address 127.5.46.126:46433)
0928 17:56:56.640191 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:56.640193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:56.640410 (+ 217us) server_negotiation.cc:407] Connection header received
0928 17:56:56.640913 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:56.640914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:56.640927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:56.640979 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:56.642221 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.642720 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.643760 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.644038 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.644121 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.646774 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:56.646775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:56.646776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:56.646785 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:56.648495 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.648891 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.648893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.648894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.648958 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.649192 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.649193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.649193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.649353 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:56.649430 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:56.649699 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 17:56:56.649794 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:56:56.649948 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.640099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:33617)
0928 17:56:56.640268 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:56.640315 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:56.640805 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:56.641065 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:56.641070 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:56.641849 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:56.642132 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.642138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.642825 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.642826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.643641 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.643643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.644132 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.644133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.644276 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.644879 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:56.644889 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:56.646668 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:56.648575 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.648581 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.648584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.648811 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.649027 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.649028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.649028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.649123 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.649430 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:56.649435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:56.649562 (+ 127us) client_negotiation.cc:769] Sending connection context
0928 17:56:56.649649 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 17:56:56.649771 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 17:56:56.650528 13821 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46433
I20260928 17:56:56.650655 13821 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:56.650844 13821 heartbeater.cc:511] Master 127.5.46.126:46433 requested a full tablet report, sending...
I20260928 17:56:56.651234 13496 ts_manager.cc:194] Registered new tserver with Master: 44ce5d0c3ee349c9b30fb29ede33b420 (127.5.46.66:42853)
I20260928 17:56:56.651612 13496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:33617
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:56:56.667107 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:56.746707 13826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:56.747011 13826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:56.747053 13826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:56.748275 13826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:56.748351 13826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:56.748396 13826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:56.749887 13826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:56.750166 13826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:56.750398 13826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:56.755837 13831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.755837 13832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.755962 13834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.756090 13826 server_base.cc:1034] running on GCE node
I20260928 17:56:56.756356 13826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:56.756568 13826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:56.757704 13826 hybrid_clock.cc:648] HybridClock initialized: now 1790618216757690 us; error 29 us; skew 500 ppm
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:56.760473 13826 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:56:56.760886 13826 webserver.cc:460] Webserver started at http://127.5.46.67:38403/ using document root <none> and password file <none>
I20260928 17:56:56.761093 13826 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:56.761135 13826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:56.761226 13826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:56:56.762362 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "a379123ceaa548468c8e86f5487d62c2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "a4de563909440dff85a0333e13f12e9a"
server_key_iv: "82a1c705edec54ab72099faf9052381f"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.762660 13826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "a379123ceaa548468c8e86f5487d62c2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "a4de563909440dff85a0333e13f12e9a"
server_key_iv: "82a1c705edec54ab72099faf9052381f"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.764040 13826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 17:56:56.764882 13841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.765077 13826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:56.765163 13826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "a379123ceaa548468c8e86f5487d62c2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "a4de563909440dff85a0333e13f12e9a"
server_key_iv: "82a1c705edec54ab72099faf9052381f"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.765223 13826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:56.772629 13826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:56.774811 13826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:56.774930 13826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:56.775111 13826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:56.775421 13826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:56.775462 13826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.775486 13826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:56.775498 13826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.781580 13826 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45967
I20260928 17:56:56.781666 13954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45967 every 8 connection(s)
I20260928 17:56:56.781998 13826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:56:56.791749 13540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.782754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58791 (local address 127.5.46.126:46433)
0928 17:56:56.782830 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:56.782832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:56.783013 (+ 181us) server_negotiation.cc:407] Connection header received
0928 17:56:56.783520 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:56.783521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:56.783531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:56.783584 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:56.784672 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.785150 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.786308 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.786507 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.786574 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.788819 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:56.788821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:56.788821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:56.788829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:56.790243 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.790695 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.790697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.790697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.790758 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:56.791023 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:56.791024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:56.791024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:56.791178 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:56.791262 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:56.791570 (+ 308us) server_negotiation.cc:299] Negotiation successful
0928 17:56:56.791651 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 17:56:56.791996 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:56.782719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:58791)
0928 17:56:56.782887 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:56.782927 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:56.783424 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:56.783644 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:56.783647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:56.784369 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:56.784588 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.784594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.785238 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.785239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.786193 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:56.786195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:56.786607 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:56.786608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:56.786723 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:56.787229 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:56.787238 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:56.788698 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:56:56.790348 (+ 1650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.790355 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.790358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.790599 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.790824 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:56:56.790825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:56:56.790826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:56:56.790947 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:56:56.791251 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:56:56.791258 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:56:56.791426 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 17:56:56.791537 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 17:56:56.791706 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260928 17:56:56.792222 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13826
I20260928 17:56:56.792339 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:56:56.792472 13955 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46433
I20260928 17:56:56.792573 5305 external_mini_cluster.cc:1428] Setting key 8ef47c13236e27d5af8a191439db04b0
I20260928 17:56:56.792567 13955 heartbeater.cc:464] Registering TS with master...
I20260928 17:56:56.792760 13955 heartbeater.cc:511] Master 127.5.46.126:46433 requested a full tablet report, sending...
I20260928 17:56:56.793181 13496 ts_manager.cc:194] Registered new tserver with Master: a379123ceaa548468c8e86f5487d62c2 (127.5.46.67:45967)
I20260928 17:56:56.793622 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:56:56.793738 13496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:58791
WARNING: no policy specified for oryx/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
Entry for principal oryx/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab.
WARNING: no policy specified for oryx/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
Entry for principal oryx/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab.
WARNING: no policy specified for oryx/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
Entry for principal oryx/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab.
I20260928 17:56:56.867345 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13558
I20260928 17:56:56.871906 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13692
I20260928 17:56:56.876183 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13826
I20260928 17:56:56.880591 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13464
I20260928 17:56:56.885887 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46433
--webserver_interface=127.5.46.126
--webserver_port=41801
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:56:56.967990 13964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:56.968185 13964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:56.968207 13964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:56.969280 13964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:56:56.969314 13964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:56.969328 13964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:56.969340 13964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:56:56.969352 13964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:56:56.970739 13964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=41801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:56.970924 13964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:56.971086 13964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:56.976122 13964 server_base.cc:1034] running on GCE node
W20260928 17:56:56.976099 13969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.976091 13970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:56.976091 13972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:56.976590 13964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:56.976904 13964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:56.978098 13964 hybrid_clock.cc:648] HybridClock initialized: now 1790618216978074 us; error 36 us; skew 500 ppm
Sep 28 17:56:56 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618216, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:56.986826 13964 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:56:56.987296 13964 webserver.cc:460] Webserver started at http://127.5.46.126:41801/ using document root <none> and password file <none>
I20260928 17:56:56.987537 13964 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:56.987596 13964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:56.989153 13964 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:56.989871 13979 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:56.990041 13964 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:56:56.990129 13964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "b00d4f7de11f46f6aefdd6affc4420e2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "7c3c988b3bbad7365bc2a980a0d05dd9"
server_key_iv: "de353395e88a1e6a1aa35cf9e0d165db"
server_key_version: "encryptionkey@0"
I20260928 17:56:56.990470 13964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:57.013512 13964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:57.015995 13964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:57.016119 13964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:57.020213 13964 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46433
I20260928 17:56:57.020402 14031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46433 every 8 connection(s)
I20260928 17:56:57.020689 13964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:56:57.021147 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 13964
I20260928 17:56:57.021507 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:39705
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:46433
--webserver_port=33277
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:56:57.023213 14032 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:56:57.023995 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Bootstrap starting.
I20260928 17:56:57.032835 14032 log.cc:824] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Log is configured to *not* fsync() on all Append() calls
I20260928 17:56:57.034860 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:56:57.035225 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Bootstrap complete.
I20260928 17:56:57.039155 14032 raft_consensus.cc:348] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:57.039541 14032 raft_consensus.cc:729] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00d4f7de11f46f6aefdd6affc4420e2, State: Initialized, Role: FOLLOWER
I20260928 17:56:57.039752 14032 consensus_queue.cc:260] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:57.039829 14032 raft_consensus.cc:388] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:56:57.039927 14032 raft_consensus.cc:482] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:56:57.039981 14032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:56:57.041121 14032 raft_consensus.cc:504] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:57.041257 14032 leader_election.cc:302] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b00d4f7de11f46f6aefdd6affc4420e2; no voters:
I20260928 17:56:57.041491 14032 leader_election.cc:288] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 17:56:57.041538 14036 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 17:56:57.041678 14036 raft_consensus.cc:686] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [term 2 LEADER]: Becoming Leader. State: Replica: b00d4f7de11f46f6aefdd6affc4420e2, State: Running, Role: LEADER
I20260928 17:56:57.041796 14032 sys_catalog.cc:564] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:56:57.041781 14036 consensus_queue.cc:237] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } }
I20260928 17:56:57.042140 14038 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } } }
I20260928 17:56:57.042140 14036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b00d4f7de11f46f6aefdd6affc4420e2. Latest consensus state: current_term: 2 leader_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00d4f7de11f46f6aefdd6affc4420e2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46433 } } }
I20260928 17:56:57.042250 14038 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:57.042269 14036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2 [sys.catalog]: This master's current role is: LEADER
I20260928 17:56:57.042455 14041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:56:57.043045 14041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:56:57.043211 14041 catalog_manager.cc:1259] Loaded cluster ID: 54df42caa2584f34b1da1f64a208d5df
I20260928 17:56:57.043435 14041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:56:57.044795 14041 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:56:57.045001 14041 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b00d4f7de11f46f6aefdd6affc4420e2: Loaded TSK: 0
I20260928 17:56:57.045226 14041 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 17:56:57.108148 14034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:57.108398 14034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:57.108440 14034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:57.109707 14034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:57.109781 14034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:57.109823 14034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:56:57.111340 14034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.65.keytab
--principal=oryx/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:39705
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=33277
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:57.111619 14034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:57.111924 14034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:57.117180 14061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.117201 14058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.117183 14059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:57.117669 14034 server_base.cc:1034] running on GCE node
I20260928 17:56:57.117884 14034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:57.118160 14034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:57.119325 14034 hybrid_clock.cc:648] HybridClock initialized: now 1790618217119298 us; error 45 us; skew 500 ppm
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618217, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:57.122258 14034 init.cc:377] Logged in from keytab as oryx/127.5.46.65@KRBTEST.COM (short username oryx)
I20260928 17:56:57.122857 14034 webserver.cc:460] Webserver started at http://127.5.46.65:33277/ using document root <none> and password file <none>
I20260928 17:56:57.123131 14034 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:57.123222 14034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:57.125145 14034 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:57.125845 14068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.126062 14034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:57.126164 14034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "3fbe388e22a14cfa87b9b42018a49f14"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "386adc76feb35a27da7c5b1d69d62f32"
server_key_iv: "39af13aa63fa48d23ff070c36d798a55"
server_key_version: "encryptionkey@0"
I20260928 17:56:57.126621 14034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:57.139813 14034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:57.142553 14034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:57.142711 14034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:57.142971 14034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:57.143365 14034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:57.143414 14034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.143445 14034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:57.143476 14034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.151077 14034 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:39705
I20260928 17:56:57.151118 14181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:39705 every 8 connection(s)
I20260928 17:56:57.151561 14034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:56:57.157097 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14034
I20260928 17:56:57.157454 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:42853
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:46433
--webserver_port=33945
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.163028 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.152380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:35233)
0928 17:56:57.152612 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.152664 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.153311 (+ 647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.153824 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.153830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.154643 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.154961 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.154968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.156171 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.156173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.157440 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.157443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.158714 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.158716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.158869 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.159551 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.159563 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.162760 (+ 3197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
W20260928 17:56:57.163298 14182 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46433 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.46.126:46433: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
W20260928 17:56:57.163460 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.152523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:35233 (local address 127.5.46.126:46433)
0928 17:56:57.152719 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.152727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.152754 (+ 27us) server_negotiation.cc:407] Connection header received
0928 17:56:57.153506 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.153512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.153557 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.153669 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.155088 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.156066 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.157769 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.158498 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.158612 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.162964 (+ 4352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:35233: BlockingRecv error: recv got EOF from 127.5.46.65:35233 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.169674 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.163570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:60741)
0928 17:56:57.163614 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.163627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.163679 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.164003 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.164005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.164182 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.164248 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.164251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.165188 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.165189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.166131 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.166134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.166624 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.166625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.166701 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.167230 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.167235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.169567 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260928 17:56:57.169920 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.163743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:60741 (local address 127.5.46.126:46433)
0928 17:56:57.163818 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.163820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.163829 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:57.163899 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.163900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.163914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.163972 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.164346 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.165096 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.166232 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.166538 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.166611 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.169684 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:60741: BlockingRecv error: recv got EOF from 127.5.46.65:60741 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.175676 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.169949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:33977)
0928 17:56:57.170222 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.170233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.170273 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.170473 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.170474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.170619 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.170655 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.170657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.171633 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.171634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.172478 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.172481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.172951 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.172951 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.173011 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.173524 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.173529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.175597 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":75,"threads_started":1}
W20260928 17:56:57.175823 14182 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:56:57.176038 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.170112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:33977 (local address 127.5.46.126:46433)
0928 17:56:57.170305 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.170307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.170316 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:57.170338 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.170339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.170349 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.170443 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.170743 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.171448 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.172571 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.172867 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.172948 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.175966 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:33977: BlockingRecv error: recv got EOF from 127.5.46.65:33977 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
W20260928 17:56:57.244421 14186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:57.244683 14186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:57.244724 14186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:57.245954 14186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:57.246027 14186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:57.246071 14186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:56:57.247560 14186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.66.keytab
--principal=oryx/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:42853
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=33945
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:57.247831 14186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:57.248108 14186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:56:57.253820 14186 server_base.cc:1034] running on GCE node
W20260928 17:56:57.253757 14193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.253782 14196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.253767 14194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:57.254148 14186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:57.254478 14186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:57.255641 14186 hybrid_clock.cc:648] HybridClock initialized: now 1790618217255619 us; error 37 us; skew 500 ppm
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618217, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:57.258379 14186 init.cc:377] Logged in from keytab as oryx/127.5.46.66@KRBTEST.COM (short username oryx)
I20260928 17:56:57.258846 14186 webserver.cc:460] Webserver started at http://127.5.46.66:33945/ using document root <none> and password file <none>
I20260928 17:56:57.259145 14186 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:57.259223 14186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:57.260726 14186 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:57.261395 14203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.261605 14186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:56:57.261680 14186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "44ce5d0c3ee349c9b30fb29ede33b420"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "29bbd35cd27a427eef2809ea07570c1f"
server_key_iv: "194d6f6d87f31b5a084fb9531063a7e5"
server_key_version: "encryptionkey@0"
I20260928 17:56:57.262069 14186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:57.275699 14186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:57.278102 14186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:57.278264 14186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:57.278527 14186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:57.278855 14186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:57.278908 14186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.278944 14186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:57.278970 14186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.285219 14186 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:42853
I20260928 17:56:57.285266 14316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:42853 every 8 connection(s)
I20260928 17:56:57.285637 14186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 17:56:57.293282 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.286418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:56431 (local address 127.5.46.126:46433)
0928 17:56:57.286466 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.286468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.286679 (+ 211us) server_negotiation.cc:407] Connection header received
0928 17:56:57.287214 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.287216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.287233 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.287294 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.288373 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.288861 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.289883 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.290074 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.290159 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.293216 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:56431: BlockingRecv error: recv got EOF from 127.5.46.66:56431 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:56:57.293354 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.286379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:56431)
0928 17:56:57.286540 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.286586 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.287107 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.287378 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.287382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.288055 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.288263 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.288268 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.288957 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.288959 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.289789 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.289791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.290171 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.290172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.290301 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.290864 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.290872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.293143 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260928 17:56:57.293548 14317 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46433 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.46.126:46433: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
I20260928 17:56:57.293709 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14186
I20260928 17:56:57.294009 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:45967
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:46433
--webserver_port=38403
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.298969 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.293819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:42785)
0928 17:56:57.293861 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.293871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.293909 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.294046 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.294047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.294204 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.294259 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.294260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.295154 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.295155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.296160 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.296163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.296505 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.296506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.296579 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.296996 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.297003 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.298881 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 17:56:57.299023 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.293878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:42785 (local address 127.5.46.126:46433)
0928 17:56:57.293923 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.293925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.293933 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:57.293958 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.293959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.293968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.294045 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.294324 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.295052 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.296250 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.296427 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.296483 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.298951 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:42785: BlockingRecv error: recv got EOF from 127.5.46.66:42785 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.303472 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.299281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:50895)
0928 17:56:57.299376 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.299387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.299438 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.299580 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.299581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.299709 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.299738 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.299739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.300380 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.300381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.300906 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.300908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.301272 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.301273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.301324 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.301773 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.301778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.303412 (+ 1634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":78}
W20260928 17:56:57.303633 14317 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:56:57.303711 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.299335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:50895 (local address 127.5.46.126:46433)
0928 17:56:57.299370 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.299371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.299422 (+ 51us) server_negotiation.cc:407] Connection header received
0928 17:56:57.299495 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.299496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.299505 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.299576 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.299812 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.300298 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.301015 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.301202 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.301276 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.303615 (+ 2339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:50895: BlockingRecv error: recv got EOF from 127.5.46.66:50895 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 17:56:57.378980 14320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:56:57.379206 14320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:56:57.379231 14320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:56:57.380760 14320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:56:57.380822 14320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:56:57.380853 14320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:56:57.382411 14320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/krb5kdc/oryx_127.5.46.67.keytab
--principal=oryx/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:45967
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=38403
--tserver_master_addrs=127.5.46.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:56:57.382680 14320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:56:57.382939 14320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:56:57.388326 14328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.388334 14326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:56:57.388405 14325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:56:57.388424 14320 server_base.cc:1034] running on GCE node
I20260928 17:56:57.388780 14320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:56:57.389071 14320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:56:57.390352 14320 hybrid_clock.cc:648] HybridClock initialized: now 1790618217390342 us; error 36 us; skew 500 ppm
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618217, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:56:57.393221 14320 init.cc:377] Logged in from keytab as oryx/127.5.46.67@KRBTEST.COM (short username oryx)
I20260928 17:56:57.393798 14320 webserver.cc:460] Webserver started at http://127.5.46.67:38403/ using document root <none> and password file <none>
I20260928 17:56:57.394068 14320 fs_manager.cc:359] Metadata directory not provided
I20260928 17:56:57.394152 14320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:56:57.395541 14320 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:56:57.396246 14335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.396457 14320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:56:57.396564 14320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "a379123ceaa548468c8e86f5487d62c2"
format_stamp: "Formatted at 2026-09-28 17:56:56 on dist-test-slave-z2t3"
server_key: "a4de563909440dff85a0333e13f12e9a"
server_key_iv: "82a1c705edec54ab72099faf9052381f"
server_key_version: "encryptionkey@0"
I20260928 17:56:57.396938 14320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:56:57.412154 14320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:56:57.414245 14320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:56:57.414433 14320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:56:57.414691 14320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:56:57.415011 14320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:56:57.415067 14320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.415102 14320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:56:57.415130 14320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:56:57.421286 14320 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45967
I20260928 17:56:57.421336 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45967 every 8 connection(s)
I20260928 17:56:57.421720 14320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 17:56:57.429611 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.422483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:40057 (local address 127.5.46.126:46433)
0928 17:56:57.422574 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.422577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.422747 (+ 170us) server_negotiation.cc:407] Connection header received
0928 17:56:57.423292 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.423293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.423303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.423363 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.424462 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.424978 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.426076 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.426325 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.426401 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.429553 (+ 3152us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:40057: BlockingRecv error: recv got EOF from 127.5.46.67:40057 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 17:56:57.429694 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.422429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:40057)
0928 17:56:57.422591 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.422628 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.423163 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.423471 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.423475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.424127 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.424341 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.424346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.425091 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.425092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.425954 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.425957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.426431 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.426432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.426562 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.427170 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.427179 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.429444 (+ 2265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":96,"threads_started":1}
W20260928 17:56:57.429980 14449 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46433 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.46.126:46433: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
I20260928 17:56:57.430289 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14320
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.435408 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.430294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:54341)
0928 17:56:57.430378 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.430394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.430450 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.430782 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.430784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.430929 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.430978 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.430980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.431666 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.431668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.432251 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.432253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.432633 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.432634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.432709 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.433246 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.433252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.435322 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260928 17:56:57.435516 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.430585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:54341 (local address 127.5.46.126:46433)
0928 17:56:57.430646 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.430648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.430674 (+ 26us) server_negotiation.cc:407] Connection header received
0928 17:56:57.430701 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.430702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.430712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.430762 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.431058 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.431579 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.432355 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.432547 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.432634 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.435449 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:54341: BlockingRecv error: recv got EOF from 127.5.46.67:54341 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:56:57.438997 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.430698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.5.46.126:46433)
0928 17:56:57.430751 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.430754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.430946 (+ 192us) server_negotiation.cc:407] Connection header received
0928 17:56:57.431018 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.431019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.431031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.431093 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.431728 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.432301 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.433204 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.433516 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.433596 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.435035 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:56:57.435040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:56:57.435041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:56:57.435053 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:56:57.437058 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:57.437545 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:57.437548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:57.437548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:57.437656 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:56:57.437872 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:56:57.437872 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:56:57.437873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:56:57.438204 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:56:57.438373 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 17:56:57.438719 (+ 346us) server_negotiation.cc:299] Negotiation successful
0928 17:56:57.438868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:57 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:57.441076 14451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:57.435691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:49897)
0928 17:56:57.435787 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:57.435797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:57.435831 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:57.436117 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:57.436118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:57.436227 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:57.436258 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.436260 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.437176 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.437177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.437937 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:57.437939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.438384 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.438385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:57.438442 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.438995 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:57.438999 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:57.440985 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":81}
W20260928 17:56:57.441123 14185 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:57.435829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:49897 (local address 127.5.46.126:46433)
0928 17:56:57.435908 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:57.435911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:57.435919 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:56:57.435938 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:57.435940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:57.435954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:57.435990 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:57.436330 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.437055 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.438013 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:57.438293 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:57.438355 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:57.441061 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:49897: BlockingRecv error: recv got EOF from 127.5.46.67:49897 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260928 17:56:57.441280 14449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:58.180922 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:58.176238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:54581)
0928 17:56:58.176383 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:58.176394 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:58.176448 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:58.176587 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:58.176588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:58.176745 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:58.176792 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.176794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.177534 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.177534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.178069 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.178071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.178466 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.178467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.178506 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.178988 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:58.178993 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:58.180846 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
W20260928 17:56:58.181134 14454 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:58.176331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:54581 (local address 127.5.46.126:46433)
0928 17:56:58.176448 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:58.176450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:58.176459 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:58.176510 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:58.176511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:58.176520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:58.176570 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:58.176893 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.177454 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.178146 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.178392 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.178450 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.181055 (+ 2605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:54581: BlockingRecv error: recv got EOF from 127.5.46.65:54581 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":67,"threads_started":1}
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:58.309687 14455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:58.304097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:37463)
0928 17:56:58.304299 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:58.304314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:58.304417 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:58.304646 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:58.304647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:58.304824 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:58.304869 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.304871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.305577 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.305578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.306125 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.306127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.306522 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.306523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.306591 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.307178 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:58.307183 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:58.309572 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
W20260928 17:56:58.309746 14454 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:58.304211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:37463 (local address 127.5.46.126:46433)
0928 17:56:58.304306 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:58.304308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:58.304398 (+ 90us) server_negotiation.cc:407] Connection header received
0928 17:56:58.304536 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:58.304537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:58.304551 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:58.304624 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:58.304956 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.305475 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.306218 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.306425 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.306520 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.309682 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:37463: BlockingRecv error: recv got EOF from 127.5.46.66:37463 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:58 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:58.446605 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:58.441639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:40967)
0928 17:56:58.441771 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:58.441801 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:58.441863 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:58.442043 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:58.442045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:58.442271 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:58.442342 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.442344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.443046 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.443047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.443681 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:58.443684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.444112 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.444113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:58.444173 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.444640 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:58.444644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:58.446534 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
W20260928 17:56:58.446728 14454 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:58.441753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:40967 (local address 127.5.46.126:46433)
0928 17:56:58.441829 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:58.441831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:58.441844 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:56:58.441938 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:58.441939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:58.441954 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:58.442037 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:58.442414 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.442959 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.443766 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:58.444029 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:58.444102 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:58.446647 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:40967: BlockingRecv error: recv got EOF from 127.5.46.67:40967 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:59.186825 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:59.181355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:44899)
0928 17:56:59.181551 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:59.181562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:59.181636 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:59.181759 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:59.181760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:59.181934 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:59.182011 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.182013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.182707 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.182708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.183397 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.183400 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.183821 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.183822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.183904 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.184519 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:59.184525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:59.186728 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
W20260928 17:56:59.186923 14458 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:59.181469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:44899 (local address 127.5.46.126:46433)
0928 17:56:59.181587 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:59.181589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:59.181618 (+ 29us) server_negotiation.cc:407] Connection header received
0928 17:56:59.181686 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:59.181687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:59.181697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:59.181753 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:59.182072 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.182582 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.183509 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.183710 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.183779 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.186848 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:44899: BlockingRecv error: recv got EOF from 127.5.46.65:44899 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:59.315452 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:59.310313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:56731)
0928 17:56:59.310473 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:59.310485 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:59.310544 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:59.310852 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:59.310854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:59.311090 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:59.311149 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.311152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.312103 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.312104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.312642 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.312644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.312988 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.312989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.313030 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.313506 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:59.313510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:59.315380 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
W20260928 17:56:59.315548 14458 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:59.310452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:56731 (local address 127.5.46.126:46433)
0928 17:56:59.310522 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:59.310524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:59.310533 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:56:59.310690 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:59.310692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:59.310707 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:59.310815 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:59.311237 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.312015 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.312726 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.312914 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.312982 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.315482 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:56731: BlockingRecv error: recv got EOF from 127.5.46.66:56731 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:56:59 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:56:59.452108 14460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:56:59.447195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:39465)
0928 17:56:59.447396 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 17:56:59.447411 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:56:59.447509 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:56:59.447656 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:56:59.447657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:56:59.447885 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:56:59.447932 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.447934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.448628 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.448630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.449201 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:56:59.449203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.449622 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.449623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:56:59.449680 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.450164 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:56:59.450169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:56:59.451989 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":71,"threads_started":1}
W20260928 17:56:59.452142 14458 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:56:59.447326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:39465 (local address 127.5.46.126:46433)
0928 17:56:59.447400 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:56:59.447402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:56:59.447490 (+ 88us) server_negotiation.cc:407] Connection header received
0928 17:56:59.447566 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:56:59.447567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:56:59.447584 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:56:59.447647 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:56:59.448021 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.448533 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.449352 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:56:59.449533 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:56:59.449622 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:56:59.452076 (+ 2454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:39465: BlockingRecv error: recv got EOF from 127.5.46.67:39465 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:00.192111 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:00.187435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:33561)
0928 17:57:00.187610 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:00.187628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:00.187688 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:00.187828 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:00.187829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:00.188040 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:00.188088 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.188090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.188794 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.188795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.189340 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.189342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.189675 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.189675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.189713 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.190212 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:00.190216 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:00.192004 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
W20260928 17:57:00.192322 14462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:00.187494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:33561 (local address 127.5.46.126:46433)
0928 17:57:00.187628 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:00.187630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:00.187687 (+ 57us) server_negotiation.cc:407] Connection header received
0928 17:57:00.187732 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:00.187733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:00.187743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:00.187825 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:00.188176 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.188705 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.189417 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.189608 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.189666 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.192152 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:33561: BlockingRecv error: recv got EOF from 127.5.46.65:33561 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:00.320850 14463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:00.315979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:53333)
0928 17:57:00.316150 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:00.316163 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:00.316253 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:00.316406 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:00.316406 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:00.316605 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:00.316652 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.316654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.317429 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.317430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.317958 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.317960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.318351 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.318352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.318410 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.318928 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:00.318933 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:00.320750 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
W20260928 17:57:00.320909 14462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:00.316078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:53333 (local address 127.5.46.126:46433)
0928 17:57:00.316170 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:00.316173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:00.316232 (+ 59us) server_negotiation.cc:407] Connection header received
0928 17:57:00.316341 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:00.316342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:00.316354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:00.316402 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:00.316738 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.317337 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.318046 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.318266 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.318351 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.320843 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:53333: BlockingRecv error: recv got EOF from 127.5.46.66:53333 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:00 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:00.458047 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:00.452762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:36853)
0928 17:57:00.452920 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:00.452930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:00.453004 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:00.453140 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:00.453140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:00.453321 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:00.453365 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.453367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.454020 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.454020 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.454557 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:00.454559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.454910 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.454911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:00.454950 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.455546 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:00.455552 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:00.457927 (+ 2375us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
W20260928 17:57:00.458314 14462 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:00.452857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:36853 (local address 127.5.46.126:46433)
0928 17:57:00.452940 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:00.452942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:00.452990 (+ 48us) server_negotiation.cc:407] Connection header received
0928 17:57:00.453075 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:00.453076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:00.453088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:00.453133 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:00.453435 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.453931 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.454646 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:00.454836 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:00.454899 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:00.458063 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:36853: BlockingRecv error: recv got EOF from 127.5.46.67:36853 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:01.197956 14465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:01.192776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:39351)
0928 17:57:01.192988 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:01.192999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:01.193120 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:01.193284 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:01.193285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:01.193445 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:01.193494 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.193496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.194218 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.194219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.194754 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.194757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.195157 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.195157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.195199 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.195708 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:01.195713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:01.197855 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":128,"threads_started":1}
W20260928 17:57:01.198019 14466 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:01.192860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:39351 (local address 127.5.46.126:46433)
0928 17:57:01.192978 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:01.192980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:01.193118 (+ 138us) server_negotiation.cc:407] Connection header received
0928 17:57:01.193178 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:01.193179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:01.193190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:01.193279 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:01.193565 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.194120 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.194859 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.195078 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.195143 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.197944 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:39351: BlockingRecv error: recv got EOF from 127.5.46.65:39351 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":50,"threads_started":1}
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:01.326455 14467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:01.321444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:44367)
0928 17:57:01.321670 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:01.321682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:01.321781 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:01.321938 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:01.321939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:01.322136 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:01.322182 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.322184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.322856 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.322857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.323390 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.323393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.323781 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.323782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.323877 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.324459 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:01.324464 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:01.326347 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":138,"threads_started":1}
W20260928 17:57:01.326508 14466 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:01.321577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:44367 (local address 127.5.46.126:46433)
0928 17:57:01.321664 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:01.321667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:01.321766 (+ 99us) server_negotiation.cc:407] Connection header received
0928 17:57:01.321844 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:01.321845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:01.321865 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:01.321933 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:01.322281 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.322773 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.323492 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.323691 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.323781 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.326442 (+ 2661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:44367: BlockingRecv error: recv got EOF from 127.5.46.66:44367 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:01 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:01.463399 14468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:01.458666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:35699)
0928 17:57:01.458841 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:01.458853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:01.458920 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:01.459068 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:01.459068 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:01.459225 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:01.459272 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.459273 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.459974 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.459975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.460527 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:01.460529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.460898 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.460899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:01.460966 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.461544 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:01.461550 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:01.463339 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":78,"threads_started":1}
W20260928 17:57:01.463479 14466 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:01.458738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:35699 (local address 127.5.46.126:46433)
0928 17:57:01.458837 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:01.458839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:01.458923 (+ 84us) server_negotiation.cc:407] Connection header received
0928 17:57:01.458972 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:01.458973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:01.458987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:01.459047 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:01.459370 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.459890 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.460603 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:01.460779 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:01.460852 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:01.463428 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:35699: BlockingRecv error: recv got EOF from 127.5.46.67:35699 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:02.218290 14469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:02.198440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:51689)
0928 17:57:02.198599 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:02.198611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:02.198673 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:02.198863 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:02.198864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:02.199028 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:02.199075 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.199077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.199762 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.199763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.200380 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.200383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.200727 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.200728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.200798 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.215985 (+ 15187us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:02.215999 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:02.218210 (+ 2211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
W20260928 17:57:02.218461 14470 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:02.198618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:51689 (local address 127.5.46.126:46433)
0928 17:57:02.198744 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:02.198746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:02.198753 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:57:02.198775 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:02.198776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:02.198786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:02.198848 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:02.199134 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.199674 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.200466 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.200664 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.200718 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.218380 (+ 17662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:51689: BlockingRecv error: recv got EOF from 127.5.46.65:51689 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":77,"threads_started":1}
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:02.331797 14471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:02.327089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:52799)
0928 17:57:02.327209 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:02.327221 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:02.327282 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:02.327415 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:02.327415 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:02.327600 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:02.327647 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.327648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.328382 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.328383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.328909 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.328911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.329286 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.329287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.329359 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.329843 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:02.329848 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:02.331719 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260928 17:57:02.332005 14470 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:02.327196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:52799 (local address 127.5.46.126:46433)
0928 17:57:02.327254 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:02.327256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:02.327264 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:57:02.327347 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:02.327348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:02.327360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:02.327409 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:02.327732 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.328292 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.328995 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.329198 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.329285 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.331855 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:52799: BlockingRecv error: recv got EOF from 127.5.46.66:52799 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:02 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:02.468953 14472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:02.464032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:47529)
0928 17:57:02.464154 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:02.464165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:02.464278 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:02.464501 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:02.464502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:02.464678 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:02.464726 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.464728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.465452 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.465453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.466026 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:02.466035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.466402 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.466403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:02.466453 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.466963 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:02.466967 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:02.468891 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
W20260928 17:57:02.469238 14470 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:02.464095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:47529 (local address 127.5.46.126:46433)
0928 17:57:02.464141 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:02.464143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:02.464254 (+ 111us) server_negotiation.cc:407] Connection header received
0928 17:57:02.464387 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:02.464388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:02.464402 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:02.464465 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:02.464820 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.465318 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.466116 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:02.466297 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:02.466401 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:02.469162 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:47529: BlockingRecv error: recv got EOF from 127.5.46.67:47529 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:03.224166 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:03.218971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:47895)
0928 17:57:03.219162 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:03.219176 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:03.219237 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:03.219418 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:03.219419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:03.219586 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:03.219638 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.219640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.220492 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.220493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.221019 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.221021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.221422 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.221423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.221486 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.222002 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:03.222007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:03.224103 (+ 2096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
W20260928 17:57:03.224340 14474 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:03.219048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:47895 (local address 127.5.46.126:46433)
0928 17:57:03.219210 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:03.219212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:03.219240 (+ 28us) server_negotiation.cc:407] Connection header received
0928 17:57:03.219306 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:03.219306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:03.219316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:03.219372 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:03.219755 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.220400 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.221106 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.221329 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.221394 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.224235 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:47895: BlockingRecv error: recv got EOF from 127.5.46.65:47895 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:03.337154 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:03.332312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:58493)
0928 17:57:03.332446 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:03.332479 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:03.332537 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:03.332727 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:03.332728 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:03.332914 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:03.332959 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.332961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.333682 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.333683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.334235 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.334238 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.334598 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.334599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.334659 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.335155 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:03.335159 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:03.337055 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":57,"threads_started":1}
W20260928 17:57:03.337217 14474 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:03.332433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:58493 (local address 127.5.46.126:46433)
0928 17:57:03.332516 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:03.332518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:03.332529 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:57:03.332615 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:03.332616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:03.332636 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:03.332727 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:03.333050 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.333583 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.334328 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.334521 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.334601 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.337152 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:58493: BlockingRecv error: recv got EOF from 127.5.46.66:58493 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:03 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:03.474641 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:03.469494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:34843)
0928 17:57:03.469678 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:03.469690 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:03.469801 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:03.469994 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:03.469995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:03.470182 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:03.470226 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.470227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.471070 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.471071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.471673 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:03.471676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.472185 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.472186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:03.472234 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.472720 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:03.472724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:03.474580 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
W20260928 17:57:03.474787 14474 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:03.469580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34843 (local address 127.5.46.126:46433)
0928 17:57:03.469660 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:03.469663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:03.469780 (+ 117us) server_negotiation.cc:407] Connection header received
0928 17:57:03.469895 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:03.469897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:03.469918 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:03.469985 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:03.470315 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.470992 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.471812 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:03.472092 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:03.472161 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:03.474705 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:34843: BlockingRecv error: recv got EOF from 127.5.46.67:34843 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:04.229700 14477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:04.224802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:42315)
0928 17:57:04.224936 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:04.224955 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:04.225008 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:04.225227 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:04.225228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:04.225398 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:04.225447 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.225449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.226290 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.226291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.226891 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.226894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.227255 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.227256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.227294 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.227738 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:04.227743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:04.229640 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260928 17:57:04.229851 14478 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:04.224919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:42315 (local address 127.5.46.126:46433)
0928 17:57:04.225058 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:04.225060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:04.225068 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:57:04.225095 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:04.225096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:04.225106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:04.225179 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:04.225536 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.226205 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.226988 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.227181 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.227245 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.229755 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:42315: BlockingRecv error: recv got EOF from 127.5.46.65:42315 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:04.342878 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:04.337723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:32959)
0928 17:57:04.337872 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:04.337886 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:04.337941 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:04.338081 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:04.338082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:04.338243 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:04.338290 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.338292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.339069 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.339070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.339616 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.339618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.340032 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.340033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.340079 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.340567 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:04.340571 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:04.342796 (+ 2225us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260928 17:57:04.343021 14478 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:04.337854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:32959 (local address 127.5.46.126:46433)
0928 17:57:04.337913 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:04.337915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:04.337940 (+ 25us) server_negotiation.cc:407] Connection header received
0928 17:57:04.337996 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:04.337996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:04.338008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:04.338072 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:04.338385 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.338949 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.339723 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.339958 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.340018 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.342932 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:32959: BlockingRecv error: recv got EOF from 127.5.46.66:32959 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:04 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:04.479827 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:04.475172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:58417)
0928 17:57:04.475349 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:04.475361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:04.475421 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:04.475538 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:04.475538 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:04.475698 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:04.475744 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.475745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.476448 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.476449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.476982 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:04.476985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.477285 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.477286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:04.477322 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.477777 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:04.477781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:04.479767 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":73,"threads_started":1}
W20260928 17:57:04.480054 14478 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:04.475277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58417 (local address 127.5.46.126:46433)
0928 17:57:04.475403 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:04.475405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:04.475415 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:57:04.475471 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:04.475471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:04.475484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:04.475533 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:04.475802 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.476379 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.477053 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:04.477225 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:04.477274 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:04.479953 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:58417: BlockingRecv error: recv got EOF from 127.5.46.67:58417 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:05.235668 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:05.230460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:48939)
0928 17:57:05.230614 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:05.230629 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:05.230702 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:05.230871 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:05.230872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:05.231050 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:05.231097 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.231099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.231928 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.231928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.232481 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.232483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.232844 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.232844 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.232891 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.233446 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:05.233451 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:05.235606 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
W20260928 17:57:05.235886 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:05.230600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:48939 (local address 127.5.46.126:46433)
0928 17:57:05.230733 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:05.230735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:05.230744 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:57:05.230774 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:05.230775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:05.230804 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:05.230864 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:05.231176 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.231792 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.232567 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.232768 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.232832 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.235765 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:48939: BlockingRecv error: recv got EOF from 127.5.46.65:48939 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:05.348688 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:05.343441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:38781)
0928 17:57:05.343631 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:05.343642 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:05.343717 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:05.343912 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:05.343916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:05.344082 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:05.344131 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.344133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.344959 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.344960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.345699 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.345702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.346126 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.346127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.346175 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.346676 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:05.346681 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:05.348592 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
W20260928 17:57:05.348737 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:05.343531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:38781 (local address 127.5.46.126:46433)
0928 17:57:05.343592 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:05.343594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:05.343721 (+ 127us) server_negotiation.cc:407] Connection header received
0928 17:57:05.343771 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:05.343772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:05.343783 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:05.343881 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:05.344201 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.344866 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.345813 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.346041 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.346097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.348673 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:38781: BlockingRecv error: recv got EOF from 127.5.46.66:38781 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:05 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:05.485293 14484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:05.480360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:40215)
0928 17:57:05.480587 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:05.480599 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:05.480699 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:05.480883 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:05.480884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:05.481061 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:05.481124 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.481126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.481806 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.481807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.482354 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:05.482355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.482728 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.482729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:05.482785 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.483274 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:05.483279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:05.485233 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
W20260928 17:57:05.485380 14482 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:05.480467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:40215 (local address 127.5.46.126:46433)
0928 17:57:05.480575 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:05.480578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:05.480685 (+ 107us) server_negotiation.cc:407] Connection header received
0928 17:57:05.480768 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:05.480769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:05.480783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:05.480869 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:05.481221 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.481722 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.482444 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:05.482653 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:05.482731 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:05.485331 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:40215: BlockingRecv error: recv got EOF from 127.5.46.67:40215 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:06.241395 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:06.236231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:42803)
0928 17:57:06.236409 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:06.236428 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:06.236486 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:06.236643 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:06.236644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:06.236808 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:06.236857 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.236858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.237538 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.237539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.238058 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.238060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.238456 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.238457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.238526 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.238999 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:06.239004 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:06.241332 (+ 2328us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
W20260928 17:57:06.241555 14486 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:06.236325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:42803 (local address 127.5.46.126:46433)
0928 17:57:06.236493 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:06.236495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:06.236506 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:57:06.236538 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:06.236539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:06.236549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:06.236621 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:06.236961 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.237462 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.238141 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.238346 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.238422 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.241464 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:42803: BlockingRecv error: recv got EOF from 127.5.46.65:42803 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:06.354133 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:06.349261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:32897)
0928 17:57:06.349412 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:06.349427 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:06.349499 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:06.349672 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:06.349673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:06.349861 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:06.349907 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.349909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.350572 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.350573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.351086 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.351088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.351454 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.351455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.351520 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.352155 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:06.352160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:06.354036 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
W20260928 17:57:06.354193 14486 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:06.349374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:32897 (local address 127.5.46.126:46433)
0928 17:57:06.349446 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:06.349449 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:06.349498 (+ 49us) server_negotiation.cc:407] Connection header received
0928 17:57:06.349571 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:06.349571 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:06.349582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:06.349663 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:06.349991 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.350479 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.351173 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.351366 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.351454 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.354129 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:32897: BlockingRecv error: recv got EOF from 127.5.46.66:32897 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:06 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:06.490955 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:06.485910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:51977)
0928 17:57:06.486125 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:06.486137 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:06.486233 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:06.486371 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:06.486372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:06.486560 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:06.486603 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.486604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.487292 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.487293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.487820 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:06.487821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.488318 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.488319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:06.488387 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.488862 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:06.488866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:06.490856 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
W20260928 17:57:06.491040 14486 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:06.486014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:51977 (local address 127.5.46.126:46433)
0928 17:57:06.486114 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:06.486116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:06.486217 (+ 101us) server_negotiation.cc:407] Connection header received
0928 17:57:06.486291 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:06.486292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:06.486308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:06.486364 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:06.486684 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.487208 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.487983 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:06.488221 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:06.488313 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:06.490958 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:51977: BlockingRecv error: recv got EOF from 127.5.46.67:51977 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:07.256194 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:07.242018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:44753)
0928 17:57:07.242159 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:07.242170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:07.242243 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:07.242415 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:07.242416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:07.242653 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:07.242728 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.242731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.243680 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.243681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.244301 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.244303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.244655 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.244656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.244696 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.253880 (+ 9184us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:07.253888 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:07.256085 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260928 17:57:07.256259 14490 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:07.242036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:44753 (local address 127.5.46.126:46433)
0928 17:57:07.242186 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:07.242188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:07.242225 (+ 37us) server_negotiation.cc:407] Connection header received
0928 17:57:07.242324 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:07.242325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:07.242336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:07.242398 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:07.242844 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.243574 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.244376 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.244583 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.244639 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.256185 (+ 11546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:44753: BlockingRecv error: recv got EOF from 127.5.46.65:44753 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":58,"threads_started":1}
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:07.359427 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:07.354650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:56441)
0928 17:57:07.354838 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:07.354850 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:07.354915 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:07.355104 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:07.355105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:07.355275 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:07.355331 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.355333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.355978 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.355979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.356508 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.356510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.356825 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.356826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.356875 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.357348 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:07.357353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:07.359314 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
W20260928 17:57:07.359457 14490 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:07.354778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:56441 (local address 127.5.46.126:46433)
0928 17:57:07.354843 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:07.354845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:07.354912 (+ 67us) server_negotiation.cc:407] Connection header received
0928 17:57:07.354962 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:07.354963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:07.354980 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:07.355028 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:07.355400 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.355915 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.356580 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.356763 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.356816 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.359410 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:56441: BlockingRecv error: recv got EOF from 127.5.46.66:56441 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:07 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:07.496594 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:07.491592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:51545)
0928 17:57:07.491779 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:07.491792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:07.491863 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:07.492018 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:07.492018 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:07.492190 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:07.492238 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.492239 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.492986 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.492987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.493551 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:07.493553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.493929 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.493930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:07.494028 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.494490 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:07.494495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:07.496441 (+ 1946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
W20260928 17:57:07.496615 14490 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:07.491753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:51545 (local address 127.5.46.126:46433)
0928 17:57:07.491878 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:07.491881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:07.491900 (+ 19us) server_negotiation.cc:407] Connection header received
0928 17:57:07.491925 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:07.491926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:07.491938 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:07.492000 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:07.492362 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.492873 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.493652 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:07.493844 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:07.493906 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:07.496545 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:51545: BlockingRecv error: recv got EOF from 127.5.46.67:51545 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:08.261648 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:08.256791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:35749)
0928 17:57:08.256989 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:08.257001 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:08.257063 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:08.257210 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:08.257211 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:08.257377 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:08.257420 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.257422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.258141 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.258142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.258683 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.258685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.259061 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.259062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.259100 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.259606 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:08.259610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:08.261535 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":106,"threads_started":1}
W20260928 17:57:08.261693 14494 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:08.256927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:35749 (local address 127.5.46.126:46433)
0928 17:57:08.257071 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:08.257072 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:08.257087 (+ 15us) server_negotiation.cc:407] Connection header received
0928 17:57:08.257117 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:08.257118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:08.257128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:08.257193 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:08.257502 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.258071 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.258782 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.258989 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.259054 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.261618 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:35749: BlockingRecv error: recv got EOF from 127.5.46.65:35749 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:08.365584 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:08.360092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:36489)
0928 17:57:08.360258 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:08.360272 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:08.360340 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:08.360529 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:08.360530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:08.360703 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:08.360747 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.360748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.361537 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.361538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.362236 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.362239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.362652 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.362653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.362702 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.363232 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:08.363238 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:08.365381 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
W20260928 17:57:08.365588 14494 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:08.360183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:36489 (local address 127.5.46.126:46433)
0928 17:57:08.360357 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:08.360359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:08.360375 (+ 16us) server_negotiation.cc:407] Connection header received
0928 17:57:08.360404 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:08.360404 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:08.360415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:08.360480 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:08.360839 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.361450 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.362337 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.362554 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.362619 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.365518 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:36489: BlockingRecv error: recv got EOF from 127.5.46.66:36489 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:08 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:08.502357 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:08.497215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:51025)
0928 17:57:08.497366 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:08.497378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:08.497448 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:08.497574 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:08.497575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:08.497775 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:08.497826 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.497828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.498507 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.498509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.499075 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:08.499077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.499537 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.499538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:08.499601 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.500095 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:08.500100 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:08.502249 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
W20260928 17:57:08.502490 14494 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:08.497276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:51025 (local address 127.5.46.126:46433)
0928 17:57:08.497328 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:08.497342 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:08.497433 (+ 91us) server_negotiation.cc:407] Connection header received
0928 17:57:08.497502 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:08.497503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:08.497518 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:08.497571 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:08.497918 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.498420 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.499171 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:08.499442 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:08.499535 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:08.502422 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:51025: BlockingRecv error: recv got EOF from 127.5.46.67:51025 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:09.267350 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:09.262224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:34195)
0928 17:57:09.262361 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:09.262374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:09.262434 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:09.262628 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:09.262629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:09.262855 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:09.262899 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.262901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.263665 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.263666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.264240 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.264242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.264716 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.264718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.264790 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.265364 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:09.265369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:09.267224 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
W20260928 17:57:09.267411 14498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:09.262270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:34195 (local address 127.5.46.126:46433)
0928 17:57:09.262398 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:09.262401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:09.262414 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:57:09.262507 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:09.262508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:09.262520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:09.262594 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:09.263014 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.263580 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.264344 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.264611 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.264713 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.267328 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:34195: BlockingRecv error: recv got EOF from 127.5.46.65:34195 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":55,"threads_started":1}
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:09.371008 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:09.366127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:54231)
0928 17:57:09.366284 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:09.366295 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:09.366365 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:09.366514 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:09.366514 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:09.366677 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:09.366721 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.366723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.367368 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.367369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.367948 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.367950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.368406 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.368407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.368459 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.368956 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:09.368961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:09.370935 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
W20260928 17:57:09.371183 14498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:09.366205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:54231 (local address 127.5.46.126:46433)
0928 17:57:09.366279 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:09.366281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:09.366352 (+ 71us) server_negotiation.cc:407] Connection header received
0928 17:57:09.366419 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:09.366420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:09.366430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:09.366494 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:09.366790 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.367288 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.368066 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.368269 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.368349 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.371083 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:54231: BlockingRecv error: recv got EOF from 127.5.46.66:54231 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:09 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:09.508015 14500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:09.502939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:34225)
0928 17:57:09.503111 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:09.503122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:09.503198 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:09.503457 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:09.503458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:09.503635 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:09.503680 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.503682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.504541 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.504542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.505221 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:09.505223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.505580 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.505581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:09.505627 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.506157 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:09.506163 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:09.507939 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
W20260928 17:57:09.508114 14498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:09.503063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34225 (local address 127.5.46.126:46433)
0928 17:57:09.503123 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:09.503125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:09.503255 (+ 130us) server_negotiation.cc:407] Connection header received
0928 17:57:09.503288 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:09.503289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:09.503303 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:09.503373 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:09.503784 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.504445 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.505301 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:09.505481 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:09.505565 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:09.508044 (+ 2479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:34225: BlockingRecv error: recv got EOF from 127.5.46.67:34225 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:10.272954 14501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:10.267960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:53697)
0928 17:57:10.268099 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:10.268118 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:10.268168 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:10.268340 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:10.268341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:10.268506 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:10.268552 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.268553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.269245 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.269246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.269825 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.269827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.270185 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.270186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.270233 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.270844 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:10.270850 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:10.272882 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
W20260928 17:57:10.273115 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:10.268022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:53697 (local address 127.5.46.126:46433)
0928 17:57:10.268154 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:10.268156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:10.268167 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:57:10.268234 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:10.268234 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:10.268245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:10.268314 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:10.268620 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.269131 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.269908 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.270105 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.270161 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.273023 (+ 2862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:53697: BlockingRecv error: recv got EOF from 127.5.46.65:53697 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":74,"threads_started":1}
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:10.376626 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:10.371567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:58087)
0928 17:57:10.371720 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:10.371732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:10.371791 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:10.371976 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:10.371977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:10.372143 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:10.372200 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.372203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.372973 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.372974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.373507 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.373508 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.373921 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.373922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.373973 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.374492 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:10.374497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:10.376549 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
W20260928 17:57:10.376734 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:10.371754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:58087 (local address 127.5.46.126:46433)
0928 17:57:10.371824 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:10.371831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:10.371840 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:57:10.371887 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:10.371887 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:10.371898 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:10.371959 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:10.372329 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.372893 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.373604 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.373820 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.373883 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.376660 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:58087: BlockingRecv error: recv got EOF from 127.5.46.66:58087 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:10 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:10.515020 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:10.508596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:48815)
0928 17:57:10.508748 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:10.508760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:10.508829 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:10.509029 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:10.509030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:10.509280 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:10.509342 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.509344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.510455 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.510457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.511356 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:10.511361 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.511964 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.511966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:10.512040 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.512648 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:10.512654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:10.514935 (+ 2281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
W20260928 17:57:10.515200 14502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:10.508679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:48815 (local address 127.5.46.126:46433)
0928 17:57:10.508755 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:10.508758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:10.508811 (+ 53us) server_negotiation.cc:407] Connection header received
0928 17:57:10.508891 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:10.508892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:10.508911 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:10.508995 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:10.509449 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.510335 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.511470 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:10.511808 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:10.511897 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:10.515042 (+ 3145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:48815: BlockingRecv error: recv got EOF from 127.5.46.67:48815 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:11.278290 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:11.273449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:35613)
0928 17:57:11.273674 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:11.273691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:11.273755 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:11.273912 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:11.273913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:11.274082 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:11.274127 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.274128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.274781 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.274782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.275309 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.275311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.275682 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.275683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.275726 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.276232 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:11.276237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:11.278190 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":122,"threads_started":1}
W20260928 17:57:11.278354 14506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:11.273595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:35613 (local address 127.5.46.126:46433)
0928 17:57:11.273741 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:11.273743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:11.273752 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:57:11.273816 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:11.273817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:11.273827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:11.273894 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:11.274191 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.274701 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.275403 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.275608 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.275667 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.278280 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:35613: BlockingRecv error: recv got EOF from 127.5.46.65:35613 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:11.382606 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:11.377249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:32877)
0928 17:57:11.377415 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:11.377427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:11.377491 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:11.377638 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:11.377639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:11.377806 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:11.377851 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.377852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.378615 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.378617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.379533 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.379536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.380066 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.380076 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.380137 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.380699 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:11.380705 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:11.382538 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":70,"threads_started":1}
W20260928 17:57:11.382709 14506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:11.377423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:32877 (local address 127.5.46.126:46433)
0928 17:57:11.377477 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:11.377481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:11.377498 (+ 17us) server_negotiation.cc:407] Connection header received
0928 17:57:11.377550 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:11.377551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:11.377562 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:11.377618 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:11.377952 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.378500 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.379626 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.379817 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.379961 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.382636 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:32877: BlockingRecv error: recv got EOF from 127.5.46.66:32877 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:11 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.67@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:11.520363 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:11.515528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.67:58693)
0928 17:57:11.515688 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:11.515700 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:11.515772 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:11.515936 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:11.515937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:11.516105 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:11.516151 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.516152 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.516823 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.516824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.517411 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:11.517414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.517780 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.517782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:11.517837 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.518325 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:11.518331 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:11.520291 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260928 17:57:11.520572 14506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:11.515596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58693 (local address 127.5.46.126:46433)
0928 17:57:11.515688 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:11.515691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:11.515781 (+ 90us) server_negotiation.cc:407] Connection header received
0928 17:57:11.515822 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:11.515822 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:11.515835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:11.515904 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:11.516221 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.516715 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.517498 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:11.517679 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:11.517731 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:11.520489 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.67:58693: BlockingRecv error: recv got EOF from 127.5.46.67:58693 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 28 17:57:12 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:12 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.65@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:12.290387 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.279031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.65:44973)
0928 17:57:12.279190 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:12.279204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:12.279284 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:12.279420 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:12.279420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:12.279592 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:12.279637 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.279639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.280353 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.280354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.280909 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.280912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.281324 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.281325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.281375 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.288144 (+ 6769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:12.288150 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:12.290314 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
W20260928 17:57:12.290587 14510 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:12.279159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:44973 (local address 127.5.46.126:46433)
0928 17:57:12.279290 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:12.279292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:12.279303 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:57:12.279327 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:12.279328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:12.279337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:12.279399 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:12.279708 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.280277 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.281035 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.281229 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.281286 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.290462 (+ 9176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.65:44973: BlockingRecv error: recv got EOF from 127.5.46.65:44973 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
Sep 28 17:57:12 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 17:57:12 dist-test-slave-z2t3 krb5kdc[13451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.46.66@KRBTEST.COM for oryx/127.5.46.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 17:57:12.388334 14511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.383266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46433 (local address 127.5.46.66:38417)
0928 17:57:12.383451 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:12.383466 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:12.383534 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:12.383685 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:12.383685 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:12.383889 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:12.383967 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.383969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.384941 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.384942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.385465 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.385467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.385813 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.385814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.385866 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.386352 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:12.386357 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:12.388275 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.46.126:46433: Server oryx/127.5.46.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
W20260928 17:57:12.388468 14510 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:12.383423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:38417 (local address 127.5.46.126:46433)
0928 17:57:12.383488 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:12.383490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:12.383526 (+ 36us) server_negotiation.cc:407] Connection header received
0928 17:57:12.383585 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:12.383586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:12.383597 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:12.383660 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:12.384063 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.384874 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.385531 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.385714 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.385770 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.388367 (+ 2597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.46.66:38417: BlockingRecv error: recv got EOF from 127.5.46.66:38417 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 17:57:12.438477 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14034
I20260928 17:57:12.443101 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14186
I20260928 17:57:12.447615 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14320
I20260928 17:57:12.452139 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 13964
2026-09-28T17:57:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17515 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T17:57:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:57:12Z Disabled control of system clock
I20260928 17:57:12.483774 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:36949
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:36949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 17:57:12.566542 14515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:12.566766 14515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:12.566787 14515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:12.568147 14515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:57:12.568192 14515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:12.568205 14515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:12.568217 14515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:57:12.568229 14515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:57:12.569640 14515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:36949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:36949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:12.569947 14515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:12.570268 14515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:57:12.575538 14515 server_base.cc:1034] running on GCE node
W20260928 17:57:12.577349 14521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:12.577656 14523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:12.577811 14520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:12.578111 14515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:12.578390 14515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:12.579543 14515 hybrid_clock.cc:648] HybridClock initialized: now 1790618232579519 us; error 35 us; skew 500 ppm
I20260928 17:57:12.580933 14515 webserver.cc:460] Webserver started at http://127.5.46.126:41155/ using document root <none> and password file <none>
I20260928 17:57:12.581195 14515 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:12.581243 14515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:12.581398 14515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:12.582566 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "1746aceb9f6e47edaa6f0d9f433849d1"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "1d91e1b665f04950465e078ee1446eef"
server_key_iv: "944bff4a6191b0c4cdb8b3cb80e4e913"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.582950 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "1746aceb9f6e47edaa6f0d9f433849d1"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "1d91e1b665f04950465e078ee1446eef"
server_key_iv: "944bff4a6191b0c4cdb8b3cb80e4e913"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.584491 14515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:57:12.585300 14529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.585635 14515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:12.585727 14515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "1746aceb9f6e47edaa6f0d9f433849d1"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "1d91e1b665f04950465e078ee1446eef"
server_key_iv: "944bff4a6191b0c4cdb8b3cb80e4e913"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.585788 14515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:12.613345 14515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:12.613641 14515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:12.613744 14515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:12.617718 14515 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:36949
I20260928 17:57:12.617774 14581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:36949 every 8 connection(s)
I20260928 17:57:12.618158 14515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:57:12.618752 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14515
I20260928 17:57:12.618865 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:57:12.618999 14582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:12.619087 5305 external_mini_cluster.cc:1428] Setting key 37bbcb9c4fda637a6c742da4cb6e44c5
I20260928 17:57:12.623028 14582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1: Bootstrap starting.
I20260928 17:57:12.623792 14582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:12.624147 14582 log.cc:824] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:12.625145 14582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1: No bootstrap required, opened a new log
I20260928 17:57:12.625334 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.619667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57864 (local address 127.5.46.126:36949)
0928 17:57:12.619877 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:12.619885 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:12.619913 (+ 28us) server_negotiation.cc:407] Connection header received
0928 17:57:12.620614 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:12.620622 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:12.620693 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:12.620821 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:12.621107 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.621972 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.623087 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.623578 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.623675 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.624500 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:12.624505 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:12.624505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:12.624717 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:12.624761 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:12.624784 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:12.624838 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:12.624969 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 17:57:12.625092 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260928 17:57:12.627134 14582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } }
I20260928 17:57:12.627305 14582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:12.627364 14582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1746aceb9f6e47edaa6f0d9f433849d1, State: Initialized, Role: FOLLOWER
I20260928 17:57:12.627490 14582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } }
I20260928 17:57:12.627588 14582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:57:12.627633 14582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:57:12.627683 14582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:12.628351 14582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } }
I20260928 17:57:12.628475 14582 leader_election.cc:302] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1746aceb9f6e47edaa6f0d9f433849d1; no voters:
I20260928 17:57:12.628664 14582 leader_election.cc:288] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:57:12.628700 14587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:57:12.628870 14587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [term 1 LEADER]: Becoming Leader. State: Replica: 1746aceb9f6e47edaa6f0d9f433849d1, State: Running, Role: LEADER
I20260928 17:57:12.628937 14582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:57:12.629004 14587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } }
I20260928 17:57:12.629361 14588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } } }
I20260928 17:57:12.629458 14588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [sys.catalog]: This master's current role is: LEADER
I20260928 17:57:12.629369 14589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1746aceb9f6e47edaa6f0d9f433849d1. Latest consensus state: current_term: 1 leader_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1746aceb9f6e47edaa6f0d9f433849d1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 36949 } } }
I20260928 17:57:12.629618 14589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1 [sys.catalog]: This master's current role is: LEADER
I20260928 17:57:12.629751 14594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:57:12.630268 14594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:57:12.631910 14594 catalog_manager.cc:1347] Generated new cluster ID: 521e802b8ea94531ae99401cffecfd3b
I20260928 17:57:12.631968 14594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:57:12.645347 14594 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:57:12.646059 14594 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:57:12.654274 14594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1746aceb9f6e47edaa6f0d9f433849d1: Generated new TSK 0
I20260928 17:57:12.654580 14594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:57:12.663817 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--builtin_ntp_servers=127.5.46.84:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 17:57:12.764743 14606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:12.764995 14606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:12.765020 14606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:12.766268 14606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:12.766332 14606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:12.766362 14606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:57:12.767800 14606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:12.768124 14606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:12.768479 14606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:12.774049 14611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:12.774083 14606 server_base.cc:1034] running on GCE node
W20260928 17:57:12.774049 14614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:12.774220 14612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:12.774488 14606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:12.774778 14606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:12.775964 14606 hybrid_clock.cc:648] HybridClock initialized: now 1790618232775936 us; error 44 us; skew 500 ppm
I20260928 17:57:12.777412 14606 webserver.cc:460] Webserver started at http://127.5.46.65:35137/ using document root <none> and password file <none>
I20260928 17:57:12.777693 14606 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:12.777781 14606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:12.777964 14606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:12.779106 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "649797d3a5a141f997e526b7663a4385"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "afb5a4ad4b61a6265b5e9d6f85b49b0b"
server_key_iv: "16ace261243a8092352638a64ad6bd00"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.779487 14606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "649797d3a5a141f997e526b7663a4385"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "afb5a4ad4b61a6265b5e9d6f85b49b0b"
server_key_iv: "16ace261243a8092352638a64ad6bd00"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.781024 14606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:57:12.782039 14620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.782235 14606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:57:12.782354 14606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "649797d3a5a141f997e526b7663a4385"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "afb5a4ad4b61a6265b5e9d6f85b49b0b"
server_key_iv: "16ace261243a8092352638a64ad6bd00"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.782486 14606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:12.795215 14606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:12.795635 14606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:12.795826 14606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:12.796190 14606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:12.796619 14606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:12.796687 14606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.796736 14606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:12.796764 14606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.803354 14606 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:45369
I20260928 17:57:12.803390 14733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:45369 every 8 connection(s)
I20260928 17:57:12.803754 14606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:57:12.809377 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.804589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:45437 (local address 127.5.46.126:36949)
0928 17:57:12.804638 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:12.804640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:12.804931 (+ 291us) server_negotiation.cc:407] Connection header received
0928 17:57:12.805501 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:12.805502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:12.805519 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:12.805569 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:12.806255 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.806765 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.807780 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.808010 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.808092 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.808998 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:12.809000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:12.809000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:12.809007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:12.809031 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:12.809036 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:12.809111 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:12.809220 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 17:57:12.809279 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:57:12.809708 14736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.804589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:36949 (local address 127.5.46.65:45437)
0928 17:57:12.804796 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:12.804832 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:12.805380 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:12.805647 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:12.805651 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:12.805659 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:12.806160 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.806164 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.806873 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.806874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.807683 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.807685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.808100 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.808101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.808246 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.808796 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:12.808797 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:12.808830 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:12.808856 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:12.809111 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:12.809119 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:12.809123 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:57:12.809225 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 17:57:12.809408 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260928 17:57:12.810199 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14606
I20260928 17:57:12.810310 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:57:12.810340 14734 heartbeater.cc:347] Connected to a master server at 127.5.46.126:36949
I20260928 17:57:12.810460 14734 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:12.810583 5305 external_mini_cluster.cc:1428] Setting key 859f8e87614b8c0c7174b745af9eb121
I20260928 17:57:12.810690 14734 heartbeater.cc:511] Master 127.5.46.126:36949 requested a full tablet report, sending...
I20260928 17:57:12.811304 14545 ts_manager.cc:194] Registered new tserver with Master: 649797d3a5a141f997e526b7663a4385 (127.5.46.65:45369)
I20260928 17:57:12.811633 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--builtin_ntp_servers=127.5.46.84:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 17:57:12.812176 14545 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.65:45437
W20260928 17:57:12.893661 14737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:12.893922 14737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:12.893962 14737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:12.895148 14737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:12.895231 14737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:12.895277 14737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:57:12.896759 14737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:12.897037 14737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:12.897349 14737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:12.902668 14742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:12.902741 14745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:12.902894 14743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:12.902926 14737 server_base.cc:1034] running on GCE node
I20260928 17:57:12.903196 14737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:12.903493 14737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:12.904657 14737 hybrid_clock.cc:648] HybridClock initialized: now 1790618232904645 us; error 30 us; skew 500 ppm
I20260928 17:57:12.905995 14737 webserver.cc:460] Webserver started at http://127.5.46.66:40633/ using document root <none> and password file <none>
I20260928 17:57:12.906245 14737 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:12.906316 14737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:12.906481 14737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:12.907616 14737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "a24f4206bd324b05b864d628b5da8e7a"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "7d3fbff9bb352adf4870b18e2a9641a5"
server_key_iv: "ecc49d584a3bd817ca9e9893fe1a7565"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.908020 14737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "a24f4206bd324b05b864d628b5da8e7a"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "7d3fbff9bb352adf4870b18e2a9641a5"
server_key_iv: "ecc49d584a3bd817ca9e9893fe1a7565"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.909384 14737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:57:12.910183 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.910470 14737 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:57:12.910586 14737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "a24f4206bd324b05b864d628b5da8e7a"
format_stamp: "Formatted at 2026-09-28 17:57:12 on dist-test-slave-z2t3"
server_key: "7d3fbff9bb352adf4870b18e2a9641a5"
server_key_iv: "ecc49d584a3bd817ca9e9893fe1a7565"
server_key_version: "encryptionkey@0"
I20260928 17:57:12.910662 14737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:12.921597 14737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:12.921973 14737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:12.922154 14737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:12.922413 14737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:12.922760 14737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:12.922827 14737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.922868 14737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:12.922899 14737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:12.928885 14737 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:45579
I20260928 17:57:12.928965 14864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:45579 every 8 connection(s)
I20260928 17:57:12.929301 14737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:57:12.934898 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.930091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:40969 (local address 127.5.46.126:36949)
0928 17:57:12.930156 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:12.930159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:12.930435 (+ 276us) server_negotiation.cc:407] Connection header received
0928 17:57:12.930944 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:12.930946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:12.930963 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:12.931042 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:12.931746 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.932381 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.933414 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.933624 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.933700 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.934601 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:12.934602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:12.934602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:12.934607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:12.934623 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:12.934628 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:12.934683 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:12.934783 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 17:57:12.934812 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:57:12.935134 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:12.930099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:36949 (local address 127.5.46.66:40969)
0928 17:57:12.930312 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:12.930346 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:12.930832 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:12.931120 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:12.931124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:12.931135 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:12.931651 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.931655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.932487 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.932488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.933303 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:12.933305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:12.933722 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:12.933723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:12.933865 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:12.934420 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:12.934421 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:12.934450 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:12.934475 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:12.934690 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:12.934696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:12.934698 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:57:12.934786 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 17:57:12.934894 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260928 17:57:12.935703 14865 heartbeater.cc:347] Connected to a master server at 127.5.46.126:36949
I20260928 17:57:12.935810 14865 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:12.936030 14865 heartbeater.cc:511] Master 127.5.46.126:36949 requested a full tablet report, sending...
I20260928 17:57:12.936462 14545 ts_manager.cc:194] Registered new tserver with Master: a24f4206bd324b05b864d628b5da8e7a (127.5.46.66:45579)
I20260928 17:57:12.936833 14545 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.66:40969
I20260928 17:57:12.937068 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14737
I20260928 17:57:12.937203 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:57:12.937448 5305 external_mini_cluster.cc:1428] Setting key 571595d3911f00f5625a9ba400bc6b8f
I20260928 17:57:12.938561 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--builtin_ntp_servers=127.5.46.84:36923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 17:57:13.021063 14868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:13.021296 14868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:13.021328 14868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:13.022581 14868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:13.022631 14868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:13.022657 14868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:57:13.024060 14868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:36923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:36949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:13.024336 14868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:13.024652 14868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:13.029816 14874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.029850 14873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.029956 14876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.030174 14868 server_base.cc:1034] running on GCE node
I20260928 17:57:13.030431 14868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:13.030680 14868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:13.031829 14868 hybrid_clock.cc:648] HybridClock initialized: now 1790618233031806 us; error 35 us; skew 500 ppm
I20260928 17:57:13.033245 14868 webserver.cc:460] Webserver started at http://127.5.46.67:33089/ using document root <none> and password file <none>
I20260928 17:57:13.033507 14868 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:13.033557 14868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:13.033694 14868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:13.034850 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "f10105a1f62246579fe56d64a50dca1d"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "e0331c60f047270b59e09a21518680ca"
server_key_iv: "25b9d943ca4ff7eb2893e51287fc78e1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.035243 14868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "f10105a1f62246579fe56d64a50dca1d"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "e0331c60f047270b59e09a21518680ca"
server_key_iv: "25b9d943ca4ff7eb2893e51287fc78e1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.036765 14868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260928 17:57:13.037655 14882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.037863 14868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:13.037972 14868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "f10105a1f62246579fe56d64a50dca1d"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "e0331c60f047270b59e09a21518680ca"
server_key_iv: "25b9d943ca4ff7eb2893e51287fc78e1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.038065 14868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:13.064384 14868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:13.064744 14868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:13.064913 14868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:13.065177 14868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:13.065518 14868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:13.065574 14868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.065615 14868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:13.065658 14868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.071460 14868 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:39175
I20260928 17:57:13.071509 14995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:39175 every 8 connection(s)
I20260928 17:57:13.071862 14868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:57:13.073962 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 14868
I20260928 17:57:13.074072 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:57:13.074267 5305 external_mini_cluster.cc:1428] Setting key ca19364ada6d0d2173cab00b7bacaae0
I20260928 17:57:13.077350 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.072638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:57473 (local address 127.5.46.126:36949)
0928 17:57:13.072706 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.072708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.072959 (+ 251us) server_negotiation.cc:407] Connection header received
0928 17:57:13.073497 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.073498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.073512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.073562 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.074289 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:13.074845 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:13.075894 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:13.076111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:13.076195 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:13.077040 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:13.077041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:13.077041 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:13.077046 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:13.077062 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:13.077067 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:13.077136 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:13.077238 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 17:57:13.077266 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 17:57:13.077623 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.072620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:36949 (local address 127.5.46.67:57473)
0928 17:57:13.072831 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:13.072868 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:13.073386 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:13.073655 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:13.073659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:13.073667 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:13.074197 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:13.074203 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:13.074953 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:13.074955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:13.075770 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:13.075772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:13.076205 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:13.076206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:13.076353 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:13.076890 (+ 537us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:13.076890 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:13.076921 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:13.076946 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:13.077136 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:13.077142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:13.077146 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:57:13.077229 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 17:57:13.077338 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260928 17:57:13.078132 14996 heartbeater.cc:347] Connected to a master server at 127.5.46.126:36949
I20260928 17:57:13.078234 14996 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:13.078424 14996 heartbeater.cc:511] Master 127.5.46.126:36949 requested a full tablet report, sending...
I20260928 17:57:13.078781 14545 ts_manager.cc:194] Registered new tserver with Master: f10105a1f62246579fe56d64a50dca1d (127.5.46.67:39175)
I20260928 17:57:13.079119 14545 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.67:57473
I20260928 17:57:13.086050 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 17:57:13.088032 14585 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:13.087464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.5.46.126:36949)
0928 17:57:13.087526 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.087528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.087651 (+ 123us) server_negotiation.cc:407] Connection header received
0928 17:57:13.087695 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.087696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.087706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.087757 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.087955 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57866: BlockingRecv error: recv got EOF from 127.0.0.1:57866 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 17:57:13.088099 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14606
I20260928 17:57:13.092743 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14737
I20260928 17:57:13.097645 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14868
I20260928 17:57:13.102582 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 14515
2026-09-28T17:57:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (647 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T17:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:57:13Z Disabled control of system clock
I20260928 17:57:13.129298 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:45975
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:45975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 17:57:13.210491 15007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:13.210722 15007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:13.210744 15007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:13.211911 15007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:57:13.211951 15007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:13.211964 15007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:13.211977 15007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:57:13.211988 15007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:57:13.213313 15007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:45975
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:45975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:13.213582 15007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:13.213908 15007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:13.218997 15012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.219062 15007 server_base.cc:1034] running on GCE node
W20260928 17:57:13.218984 15013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.218984 15015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.219481 15007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:13.219818 15007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:13.221061 15007 hybrid_clock.cc:648] HybridClock initialized: now 1790618233221020 us; error 29 us; skew 500 ppm
I20260928 17:57:13.222566 15007 webserver.cc:460] Webserver started at http://127.5.46.126:35029/ using document root <none> and password file <none>
I20260928 17:57:13.222832 15007 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:13.222888 15007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:13.223070 15007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:13.224354 15007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "13f6c1173c1a4b07b3fa879f460df0e8"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "81164f2887ba149f4a94b794cb418883"
server_key_iv: "f7cd757bd35505393227f2650e69a672"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.224730 15007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "13f6c1173c1a4b07b3fa879f460df0e8"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "81164f2887ba149f4a94b794cb418883"
server_key_iv: "f7cd757bd35505393227f2650e69a672"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.226064 15007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 17:57:13.226843 15021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.227033 15007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:57:13.227123 15007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "13f6c1173c1a4b07b3fa879f460df0e8"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "81164f2887ba149f4a94b794cb418883"
server_key_iv: "f7cd757bd35505393227f2650e69a672"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.227195 15007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:13.230715 15007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:13.231009 15007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:13.231158 15007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:13.235258 15007 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:45975
I20260928 17:57:13.235292 15073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:45975 every 8 connection(s)
I20260928 17:57:13.235673 15007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:57:13.236467 15074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:13.239316 15074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8: Bootstrap starting.
I20260928 17:57:13.240044 15074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:13.240355 15074 log.cc:824] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:13.241212 15074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8: No bootstrap required, opened a new log
I20260928 17:57:13.242954 15074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } }
I20260928 17:57:13.243119 15074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:13.243144 15074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f6c1173c1a4b07b3fa879f460df0e8, State: Initialized, Role: FOLLOWER
I20260928 17:57:13.243309 15074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } }
I20260928 17:57:13.243389 15074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:57:13.243414 15074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:57:13.243464 15074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:13.244067 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15007
I20260928 17:57:13.244148 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:57:13.244200 15074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } }
I20260928 17:57:13.244328 15074 leader_election.cc:302] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13f6c1173c1a4b07b3fa879f460df0e8; no voters:
I20260928 17:57:13.244385 5305 external_mini_cluster.cc:1428] Setting key ab3c6502ad903eb560be9dbee16ba2a9
I20260928 17:57:13.244532 15074 leader_election.cc:288] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:57:13.244566 15077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:57:13.244686 15077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [term 1 LEADER]: Becoming Leader. State: Replica: 13f6c1173c1a4b07b3fa879f460df0e8, State: Running, Role: LEADER
I20260928 17:57:13.244771 15077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } }
I20260928 17:57:13.244838 15074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:57:13.245364 15079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13f6c1173c1a4b07b3fa879f460df0e8. Latest consensus state: current_term: 1 leader_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } } }
I20260928 17:57:13.245464 15079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [sys.catalog]: This master's current role is: LEADER
I20260928 17:57:13.245627 15078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13f6c1173c1a4b07b3fa879f460df0e8" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45975 } } }
I20260928 17:57:13.245698 15078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8 [sys.catalog]: This master's current role is: LEADER
I20260928 17:57:13.245839 15088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:57:13.246541 15088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:57:13.247675 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.245095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.5.46.126:45975)
0928 17:57:13.245338 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.245349 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.245422 (+ 73us) server_negotiation.cc:407] Connection header received
0928 17:57:13.245944 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.245956 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.246028 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.246500 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.247125 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:13.247128 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:13.247129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:13.247132 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:13.247171 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:13.247186 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:13.247255 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:13.247423 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 17:57:13.247490 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260928 17:57:13.248529 15088 catalog_manager.cc:1347] Generated new cluster ID: b0e0a3bd083c46aeb627214e5a2669d4
I20260928 17:57:13.248584 15088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:57:13.255105 15088 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:57:13.255297 15088 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:57:13.265245 15088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13f6c1173c1a4b07b3fa879f460df0e8: Generated new TSK 0
I20260928 17:57:13.265576 15088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:57:13.268807 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--builtin_ntp_servers=127.5.46.84:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 17:57:13.351517 15098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:13.351771 15098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:13.351823 15098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:13.353111 15098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:13.353185 15098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:13.353231 15098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:57:13.354671 15098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:13.354966 15098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:13.355289 15098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:13.361445 15104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.361464 15103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.361480 15106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.361748 15098 server_base.cc:1034] running on GCE node
I20260928 17:57:13.361977 15098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:13.362241 15098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:13.363483 15098 hybrid_clock.cc:648] HybridClock initialized: now 1790618233363458 us; error 40 us; skew 500 ppm
I20260928 17:57:13.365020 15098 webserver.cc:460] Webserver started at http://127.5.46.65:36793/ using document root <none> and password file <none>
I20260928 17:57:13.365278 15098 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:13.365324 15098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:13.365468 15098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:13.366586 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "b7f8763e48ba4fd3b83c5dcd3de86720"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "c758725162dcc66fc34f54db73aa8d01"
server_key_iv: "67b512ad1fd4fdd3bbee071e2ec21d47"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.366959 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "b7f8763e48ba4fd3b83c5dcd3de86720"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "c758725162dcc66fc34f54db73aa8d01"
server_key_iv: "67b512ad1fd4fdd3bbee071e2ec21d47"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.368403 15098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:57:13.369179 15112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.369350 15098 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:57:13.369434 15098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "b7f8763e48ba4fd3b83c5dcd3de86720"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "c758725162dcc66fc34f54db73aa8d01"
server_key_iv: "67b512ad1fd4fdd3bbee071e2ec21d47"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.369531 15098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:13.373055 15098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:13.373345 15098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:13.373478 15098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:13.373726 15098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:13.374106 15098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:13.374143 15098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.374166 15098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:13.374207 15098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.380903 15098 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:43801
I20260928 17:57:13.381011 15225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:43801 every 8 connection(s)
I20260928 17:57:13.381356 15098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:57:13.383956 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15098
I20260928 17:57:13.384045 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:57:13.384256 5305 external_mini_cluster.cc:1428] Setting key ed72587b48f6ec45e9657ef15980a72b
I20260928 17:57:13.384884 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.382200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:50617 (local address 127.5.46.126:45975)
0928 17:57:13.382274 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.382276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.382554 (+ 278us) server_negotiation.cc:407] Connection header received
0928 17:57:13.383054 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.383055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.383062 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.383112 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.384597 (+ 1485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:13.384599 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:13.384599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:13.384605 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:13.384622 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:13.384627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:13.384710 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:13.384793 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 17:57:13.384815 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 17:57:13.385093 15228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.382196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:45975 (local address 127.5.46.65:50617)
0928 17:57:13.382424 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:13.382461 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:13.382956 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:13.383178 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:13.383182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:13.383190 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:13.384368 (+ 1178us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:13.384369 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:13.384435 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:13.384479 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:13.384708 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:13.384714 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:13.384717 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:57:13.384800 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 17:57:13.384868 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260928 17:57:13.385315 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--builtin_ntp_servers=127.5.46.84:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 17:57:13.385646 15226 heartbeater.cc:347] Connected to a master server at 127.5.46.126:45975
I20260928 17:57:13.385751 15226 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:13.385912 15226 heartbeater.cc:511] Master 127.5.46.126:45975 requested a full tablet report, sending...
I20260928 17:57:13.386433 15038 ts_manager.cc:194] Registered new tserver with Master: b7f8763e48ba4fd3b83c5dcd3de86720 (127.5.46.65:43801)
W20260928 17:57:13.468936 15229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:13.469180 15229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:13.469206 15229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:13.470467 15229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:13.470520 15229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:13.470549 15229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:57:13.471999 15229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:13.472283 15229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:13.472605 15229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:13.477878 15235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.477883 15234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.477898 15237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.478113 15229 server_base.cc:1034] running on GCE node
I20260928 17:57:13.478521 15229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:13.478765 15229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:13.479919 15229 hybrid_clock.cc:648] HybridClock initialized: now 1790618233479905 us; error 44 us; skew 500 ppm
I20260928 17:57:13.481379 15229 webserver.cc:460] Webserver started at http://127.5.46.66:43771/ using document root <none> and password file <none>
I20260928 17:57:13.481698 15229 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:13.481770 15229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:13.481904 15229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:13.483111 15229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "9afa41a847cb4005abecb36712d83f80"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "a4b2b9a613847b9d4f9d957ccbacda52"
server_key_iv: "1aacd2425d849dd19e733ca649f3ccd1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.483546 15229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "9afa41a847cb4005abecb36712d83f80"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "a4b2b9a613847b9d4f9d957ccbacda52"
server_key_iv: "1aacd2425d849dd19e733ca649f3ccd1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.485105 15229 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:57:13.485898 15243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.486143 15229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:13.486229 15229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "9afa41a847cb4005abecb36712d83f80"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "a4b2b9a613847b9d4f9d957ccbacda52"
server_key_iv: "1aacd2425d849dd19e733ca649f3ccd1"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.486294 15229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:13.489985 15229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:13.490257 15229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:13.490422 15229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:13.490617 15229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:13.490881 15229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:13.490911 15229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.490932 15229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:13.490945 15229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.497092 15229 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:45321
I20260928 17:57:13.497133 15356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:45321 every 8 connection(s)
I20260928 17:57:13.497484 15229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:57:13.500626 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15229
I20260928 17:57:13.500705 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.498272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:54603 (local address 127.5.46.126:45975)
0928 17:57:13.498325 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.498328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.498547 (+ 219us) server_negotiation.cc:407] Connection header received
0928 17:57:13.499078 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.499079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.499087 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.499146 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.500416 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:13.500417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:13.500418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:13.500422 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:13.500440 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:13.500445 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:13.500518 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:13.500617 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 17:57:13.500633 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:57:13.500757 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:57:13.500924 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.498238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:45975 (local address 127.5.46.66:54603)
0928 17:57:13.498409 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:13.498449 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:13.498959 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:13.499226 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:13.499231 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:13.499240 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:13.500241 (+ 1001us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:13.500242 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:13.500272 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:13.500302 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:13.500515 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:13.500521 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:13.500525 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:57:13.500629 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 17:57:13.500699 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260928 17:57:13.500962 5305 external_mini_cluster.cc:1428] Setting key 8e98938c39ae51b765b7bf56e186f078
I20260928 17:57:13.501587 15357 heartbeater.cc:347] Connected to a master server at 127.5.46.126:45975
I20260928 17:57:13.501698 15357 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:13.501853 15357 heartbeater.cc:511] Master 127.5.46.126:45975 requested a full tablet report, sending...
I20260928 17:57:13.501986 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--builtin_ntp_servers=127.5.46.84:41743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260928 17:57:13.502230 15038 ts_manager.cc:194] Registered new tserver with Master: 9afa41a847cb4005abecb36712d83f80 (127.5.46.66:45321)
W20260928 17:57:13.584254 15360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:13.584488 15360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:13.584513 15360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:13.585712 15360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:13.585753 15360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:13.585780 15360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:57:13.587121 15360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:41743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:45975
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:13.587435 15360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:13.587762 15360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:13.593253 15365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:13.593272 15368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.593384 15360 server_base.cc:1034] running on GCE node
W20260928 17:57:13.593453 15366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:13.593874 15360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:13.594137 15360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:13.595319 15360 hybrid_clock.cc:648] HybridClock initialized: now 1790618233595279 us; error 49 us; skew 500 ppm
I20260928 17:57:13.596873 15360 webserver.cc:460] Webserver started at http://127.5.46.67:37767/ using document root <none> and password file <none>
I20260928 17:57:13.597148 15360 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:13.597224 15360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:13.597378 15360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:13.598513 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "1ba5fdbbcc504b66beda14d158b7d64e"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "4e25cc192c32c1887d86013ec745c4e8"
server_key_iv: "d9948eb60235bf43a9210a143ee6726a"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.598897 15360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "1ba5fdbbcc504b66beda14d158b7d64e"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "4e25cc192c32c1887d86013ec745c4e8"
server_key_iv: "d9948eb60235bf43a9210a143ee6726a"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.600368 15360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:57:13.601207 15374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.601400 15360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:13.601512 15360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "1ba5fdbbcc504b66beda14d158b7d64e"
format_stamp: "Formatted at 2026-09-28 17:57:13 on dist-test-slave-z2t3"
server_key: "4e25cc192c32c1887d86013ec745c4e8"
server_key_iv: "d9948eb60235bf43a9210a143ee6726a"
server_key_version: "encryptionkey@0"
I20260928 17:57:13.601591 15360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:13.605090 15360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:13.605378 15360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:13.605530 15360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:13.605741 15360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:13.606047 15360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:13.606101 15360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.606141 15360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:13.606169 15360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:13.611742 15360 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45525
I20260928 17:57:13.611829 15487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45525 every 8 connection(s)
I20260928 17:57:13.612257 15360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:57:13.615350 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.613090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:53385 (local address 127.5.46.126:45975)
0928 17:57:13.613173 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.613175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.613425 (+ 250us) server_negotiation.cc:407] Connection header received
0928 17:57:13.613878 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.613879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.613884 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.613940 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.615141 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:13.615143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:13.615143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 17:57:13.615147 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:13.615162 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 17:57:13.615167 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:13.615218 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:13.615281 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 17:57:13.615294 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 17:57:13.615677 15490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:13.613070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:45975 (local address 127.5.46.67:53385)
0928 17:57:13.613288 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:13.613325 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:13.613793 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:13.614024 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:13.614028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:13.614036 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:13.614968 (+ 932us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 17:57:13.614969 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:13.614999 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 17:57:13.615043 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:13.615215 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:13.615218 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:13.615220 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:57:13.615274 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 17:57:13.615328 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":99,"threads_started":1}
I20260928 17:57:13.616215 15488 heartbeater.cc:347] Connected to a master server at 127.5.46.126:45975
I20260928 17:57:13.616312 15488 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:13.616428 15488 heartbeater.cc:511] Master 127.5.46.126:45975 requested a full tablet report, sending...
I20260928 17:57:13.616701 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15360
I20260928 17:57:13.616792 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:57:13.616869 15038 ts_manager.cc:194] Registered new tserver with Master: 1ba5fdbbcc504b66beda14d158b7d64e (127.5.46.67:45525)
I20260928 17:57:13.617039 5305 external_mini_cluster.cc:1428] Setting key 640fe6330618eba257ac2b14ed6feec2
I20260928 17:57:13.618000 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 17:57:13.619750 15082 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:57:13.619306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.5.46.126:45975)
0928 17:57:13.619381 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:13.619383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:13.619524 (+ 141us) server_negotiation.cc:407] Connection header received
0928 17:57:13.619553 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:13.619554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:13.619559 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:13.619607 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:13.619703 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46468: BlockingRecv error: recv got EOF from 127.0.0.1:46468 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 17:57:13.619823 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15098
I20260928 17:57:13.624516 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15229
I20260928 17:57:13.629077 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15360
I20260928 17:57:13.633256 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15007
2026-09-28T17:57:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (530 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:57:13 dist-test-slave-z2t3 krb5kdc[15498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:57:13 dist-test-slave-z2t3 krb5kdc[15498](info): set up 2 sockets
Sep 28 17:57:13 dist-test-slave-z2t3 krb5kdc[15498](info): commencing operation
krb5kdc: starting...
W20260928 17:57:14.688944 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:57:14 dist-test-slave-z2t3 krb5kdc[15498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:57:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:57:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:57:14.816608 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41083
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:41083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:57:14.902530 15511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:14.902822 15511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:14.902868 15511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:14.904173 15511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:57:14.904258 15511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:14.904281 15511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:14.904307 15511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:57:14.904333 15511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:57:14.905949 15511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:41083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:41083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:14.906275 15511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:14.906531 15511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:14.911720 15516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:14.911687 15519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:14.911720 15517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:14.912384 15511 server_base.cc:1034] running on GCE node
I20260928 17:57:14.912679 15511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:14.913012 15511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:14.914197 15511 hybrid_clock.cc:648] HybridClock initialized: now 1790618234914169 us; error 39 us; skew 500 ppm
Sep 28 17:57:14 dist-test-slave-z2t3 krb5kdc[15498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618234, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:57:14.916992 15511 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:57:14.917493 15511 webserver.cc:460] Webserver started at http://127.5.46.126:39005/ using document root <none> and password file <none>
I20260928 17:57:14.917739 15511 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:14.917809 15511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:14.917968 15511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:14.919036 15511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "e547fefbf67440ef8c54f7ce732510d7"
format_stamp: "Formatted at 2026-09-28 17:57:14 on dist-test-slave-z2t3"
server_key: "c721926754ae47e64e8b5453696c0b27"
server_key_iv: "e56c09beeeed2b371bc482b3b34e5c16"
server_key_version: "encryptionkey@0"
I20260928 17:57:14.919456 15511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "e547fefbf67440ef8c54f7ce732510d7"
format_stamp: "Formatted at 2026-09-28 17:57:14 on dist-test-slave-z2t3"
server_key: "c721926754ae47e64e8b5453696c0b27"
server_key_iv: "e56c09beeeed2b371bc482b3b34e5c16"
server_key_version: "encryptionkey@0"
I20260928 17:57:14.920997 15511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:57:14.921784 15526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:14.921965 15511 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:57:14.922079 15511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e547fefbf67440ef8c54f7ce732510d7"
format_stamp: "Formatted at 2026-09-28 17:57:14 on dist-test-slave-z2t3"
server_key: "c721926754ae47e64e8b5453696c0b27"
server_key_iv: "e56c09beeeed2b371bc482b3b34e5c16"
server_key_version: "encryptionkey@0"
I20260928 17:57:14.922165 15511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:14.934253 15511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:14.936437 15511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:14.936620 15511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:14.940853 15511 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:41083
I20260928 17:57:14.940907 15578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:41083 every 8 connection(s)
I20260928 17:57:14.941306 15511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:57:14.941737 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15511
I20260928 17:57:14.941840 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:57:14.942058 5305 external_mini_cluster.cc:1428] Setting key ed0bb84d7e846dcc64a17e794346210d
I20260928 17:57:14.942516 15579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:14.945420 15579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: Bootstrap starting.
I20260928 17:57:14.946053 15579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:14.946342 15579 log.cc:824] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:14.947042 15579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: No bootstrap required, opened a new log
Sep 28 17:57:14 dist-test-slave-z2t3 krb5kdc[15498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:57:14.949573 15579 raft_consensus.cc:348] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } }
I20260928 17:57:14.949787 15579 raft_consensus.cc:374] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:14.949834 15579 raft_consensus.cc:729] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e547fefbf67440ef8c54f7ce732510d7, State: Initialized, Role: FOLLOWER
I20260928 17:57:14.950007 15579 consensus_queue.cc:260] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } }
I20260928 17:57:14.950099 15579 raft_consensus.cc:388] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:57:14.950125 15579 raft_consensus.cc:482] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:57:14.950197 15579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:14.950911 15579 raft_consensus.cc:504] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } }
I20260928 17:57:14.951002 15579 leader_election.cc:302] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e547fefbf67440ef8c54f7ce732510d7; no voters:
I20260928 17:57:14.951215 15579 leader_election.cc:288] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:57:14.951257 15584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:57:14.951421 15584 raft_consensus.cc:686] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [term 1 LEADER]: Becoming Leader. State: Replica: e547fefbf67440ef8c54f7ce732510d7, State: Running, Role: LEADER
I20260928 17:57:14.951503 15579 sys_catalog.cc:564] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:57:14.951574 15584 consensus_queue.cc:237] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } }
I20260928 17:57:14.952008 15584 sys_catalog.cc:455] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e547fefbf67440ef8c54f7ce732510d7. Latest consensus state: current_term: 1 leader_uuid: "e547fefbf67440ef8c54f7ce732510d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } } }
I20260928 17:57:14.952018 15585 sys_catalog.cc:455] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e547fefbf67440ef8c54f7ce732510d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e547fefbf67440ef8c54f7ce732510d7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 41083 } } }
I20260928 17:57:14.952209 15585 sys_catalog.cc:458] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [sys.catalog]: This master's current role is: LEADER
I20260928 17:57:14.952442 15584 sys_catalog.cc:458] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7 [sys.catalog]: This master's current role is: LEADER
W20260928 17:57:14.953305 15600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:57:14.953384 15600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:57:14.953485 15593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:57:14.953668 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:14.942643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.5.46.126:41083)
0928 17:57:14.942836 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:14.942845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:14.942872 (+ 27us) server_negotiation.cc:407] Connection header received
0928 17:57:14.943310 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:14.943317 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:14.943365 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:14.943524 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:14.944239 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:14.945132 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:14.945956 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:14.946475 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:14.946601 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:14.949382 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:14.949387 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:14.949387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:57:14.949400 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:14.951462 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:14.951954 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:14.951957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:14.951958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:14.952079 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:14.952304 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:14.952304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:14.952305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:14.952617 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:14.952801 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:14.953220 (+ 419us) server_negotiation.cc:299] Negotiation successful
0928 17:57:14.953365 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260928 17:57:14.953688 15593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:57:14.955291 15593 catalog_manager.cc:1347] Generated new cluster ID: 7f6248d5d036423094d7caff9195bb3e
I20260928 17:57:14.955348 15593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:57:14.967329 15593 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:57:14.968003 15593 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:57:14.977180 15593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: Generated new TSK 0
I20260928 17:57:14.977414 15593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:57:15.008455 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:57:15.094271 15605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:15.094516 15605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:15.094542 15605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:15.095752 15605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:15.095806 15605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:15.095889 15605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:57:15.097448 15605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:15.097720 15605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:15.097981 15605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:15.103291 15611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:15.103286 15610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:15.103286 15613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:15.103395 15605 server_base.cc:1034] running on GCE node
I20260928 17:57:15.103781 15605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:15.104153 15605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:15.105314 15605 hybrid_clock.cc:648] HybridClock initialized: now 1790618235105288 us; error 36 us; skew 500 ppm
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:57:15.108695 15605 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:57:15.109234 15605 webserver.cc:460] Webserver started at http://127.5.46.65:36071/ using document root <none> and password file <none>
I20260928 17:57:15.109496 15605 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:15.109573 15605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:15.109731 15605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:15.110812 15605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "b7a43b25a9954bdcaeefb6a7010d5eec"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "d676c2cfa113a72f8cacfeed2b707183"
server_key_iv: "10f6f425bbd9073c05284ecc92a0b5d7"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.111189 15605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "b7a43b25a9954bdcaeefb6a7010d5eec"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "d676c2cfa113a72f8cacfeed2b707183"
server_key_iv: "10f6f425bbd9073c05284ecc92a0b5d7"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.112692 15605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:57:15.113677 15620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.113909 15605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:57:15.113999 15605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "b7a43b25a9954bdcaeefb6a7010d5eec"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "d676c2cfa113a72f8cacfeed2b707183"
server_key_iv: "10f6f425bbd9073c05284ecc92a0b5d7"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.114060 15605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:15.123497 15605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:15.125681 15605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:15.125849 15605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:15.126123 15605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:15.126391 15605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:15.126422 15605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.126444 15605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:15.126456 15605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.132341 15605 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:34233
I20260928 17:57:15.132385 15733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:34233 every 8 connection(s)
I20260928 17:57:15.132745 15605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:57:15.134181 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15605
I20260928 17:57:15.134271 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:57:15.134522 5305 external_mini_cluster.cc:1428] Setting key fc5ce8e58b398d05a686d4c7015a5ba9
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
I20260928 17:57:15.144310 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.133537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:47389 (local address 127.5.46.126:41083)
0928 17:57:15.133607 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.133609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.133845 (+ 236us) server_negotiation.cc:407] Connection header received
0928 17:57:15.134386 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.134388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.134400 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.134446 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:15.135735 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.136327 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.137894 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.138117 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.138208 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.140884 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:15.140885 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:15.140885 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:57:15.140892 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:15.142445 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.143040 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.143040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.143041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.143111 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.143443 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.143445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.143445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.143598 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:15.143665 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.144093 (+ 428us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.144179 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 17:57:15.144418 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.133528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41083 (local address 127.5.46.65:47389)
0928 17:57:15.133717 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.133755 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.134267 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.134530 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.134534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.135401 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:15.135641 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.135646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.136424 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.136425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.137804 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.137807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.138220 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.138221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.138359 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.138931 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:15.138940 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:15.140777 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:15.142543 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.142551 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.142555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.142951 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.143182 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.143183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.143183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.143339 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.143677 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:15.143684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:15.143923 (+ 239us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.144040 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.144176 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260928 17:57:15.145035 15734 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41083
I20260928 17:57:15.145164 15734 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:15.145462 15734 heartbeater.cc:511] Master 127.5.46.126:41083 requested a full tablet report, sending...
I20260928 17:57:15.146116 15543 ts_manager.cc:194] Registered new tserver with Master: b7a43b25a9954bdcaeefb6a7010d5eec (127.5.46.65:34233)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
I20260928 17:57:15.146991 15543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:47389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:57:15.163775 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:57:15.246779 15739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:15.247002 15739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:15.247030 15739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:15.248194 15739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:15.248234 15739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:15.248261 15739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:57:15.249684 15739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:15.249891 15739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:15.250077 15739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:15.255677 15744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:15.255677 15747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:15.255684 15745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:15.256016 15739 server_base.cc:1034] running on GCE node
I20260928 17:57:15.256242 15739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:15.256501 15739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:15.257687 15739 hybrid_clock.cc:648] HybridClock initialized: now 1790618235257677 us; error 40 us; skew 500 ppm
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:57:15.260463 15739 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:57:15.261116 15739 webserver.cc:460] Webserver started at http://127.5.46.66:36649/ using document root <none> and password file <none>
I20260928 17:57:15.261525 15739 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:15.261598 15739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:15.261745 15739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:15.262872 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "df9c67b68ebc4a7e97bc3648d53c03f0"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "17a73386cabaea3089c407a0d52a0cc3"
server_key_iv: "873b887771faf6eeccd281e55cfd0280"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.263252 15739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "df9c67b68ebc4a7e97bc3648d53c03f0"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "17a73386cabaea3089c407a0d52a0cc3"
server_key_iv: "873b887771faf6eeccd281e55cfd0280"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.264787 15739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:57:15.265585 15754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.265784 15739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:15.265879 15739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "df9c67b68ebc4a7e97bc3648d53c03f0"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "17a73386cabaea3089c407a0d52a0cc3"
server_key_iv: "873b887771faf6eeccd281e55cfd0280"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.265954 15739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:15.295281 15739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:15.297420 15739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:15.297571 15739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:15.297844 15739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:15.298177 15739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:15.298234 15739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.298270 15739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:15.298296 15739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.304374 15739 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:41269
I20260928 17:57:15.304399 15867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:41269 every 8 connection(s)
I20260928 17:57:15.304791 15739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:57:15.309085 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15739
I20260928 17:57:15.309216 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:57:15.309397 5305 external_mini_cluster.cc:1428] Setting key 3d8d19ace090c01aa3ee2d8aff0026e9
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:57:15.315439 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.305557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41083 (local address 127.5.46.66:36273)
0928 17:57:15.305733 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.305778 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.306268 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.306504 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.306507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.307100 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:15.307306 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.307312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.308106 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.308108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.309041 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.309044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.309567 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.309569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.309712 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.310329 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:15.310339 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:15.312079 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:15.314027 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.314035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.314037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.314282 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.314513 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.314514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.314514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.314618 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.314920 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:15.314924 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:15.315055 (+ 131us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.315140 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.315266 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260928 17:57:15.315443 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.305594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:36273 (local address 127.5.46.126:41083)
0928 17:57:15.305642 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.305643 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.305864 (+ 221us) server_negotiation.cc:407] Connection header received
0928 17:57:15.306371 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.306372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.306381 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.306445 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:15.307399 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.307986 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.309175 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.309441 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.309537 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.312177 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:15.312180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:15.312180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:57:15.312191 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:15.313917 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.314369 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.314371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.314371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.314441 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.314690 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.314691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.314692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.314847 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:15.314929 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.315190 (+ 261us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.315294 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:57:15.316087 15868 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41083
I20260928 17:57:15.316185 15868 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:15.316371 15868 heartbeater.cc:511] Master 127.5.46.126:41083 requested a full tablet report, sending...
I20260928 17:57:15.316872 15543 ts_manager.cc:194] Registered new tserver with Master: df9c67b68ebc4a7e97bc3648d53c03f0 (127.5.46.66:41269)
I20260928 17:57:15.317273 15543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:36273
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:57:15.339314 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:57:15.424186 15873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:57:15.424400 15873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:57:15.424461 15873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:57:15.425734 15873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:57:15.425779 15873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:57:15.425808 15873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:57:15.427310 15873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:41083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:57:15.427518 15873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:57:15.427713 15873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:57:15.432917 15879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:15.433075 15873 server_base.cc:1034] running on GCE node
W20260928 17:57:15.432894 15881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:57:15.432894 15878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:57:15.433408 15873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:57:15.433719 15873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:57:15.434870 15873 hybrid_clock.cc:648] HybridClock initialized: now 1790618235434849 us; error 41 us; skew 500 ppm
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:57:15.437587 15873 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:57:15.438055 15873 webserver.cc:460] Webserver started at http://127.5.46.67:36323/ using document root <none> and password file <none>
I20260928 17:57:15.438302 15873 fs_manager.cc:359] Metadata directory not provided
I20260928 17:57:15.438350 15873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:57:15.438483 15873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:57:15.439553 15873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "04d85110aee94e0780c761b458eb286d"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "f3b4b2d4b1e96a3f0502e83e9dcd40ee"
server_key_iv: "22c271f1415a096c515cf8626cb5b6a0"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.439965 15873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "04d85110aee94e0780c761b458eb286d"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "f3b4b2d4b1e96a3f0502e83e9dcd40ee"
server_key_iv: "22c271f1415a096c515cf8626cb5b6a0"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.441322 15873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:15.442113 15888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.442292 15873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:57:15.442381 15873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "04d85110aee94e0780c761b458eb286d"
format_stamp: "Formatted at 2026-09-28 17:57:15 on dist-test-slave-z2t3"
server_key: "f3b4b2d4b1e96a3f0502e83e9dcd40ee"
server_key_iv: "22c271f1415a096c515cf8626cb5b6a0"
server_key_version: "encryptionkey@0"
I20260928 17:57:15.442436 15873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:57:15.455931 15873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:57:15.458289 15873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:57:15.458415 15873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:57:15.458626 15873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:57:15.458992 15873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:57:15.459028 15873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.459051 15873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:57:15.459100 15873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:57:15.465185 15873 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:44627
I20260928 17:57:15.465224 16001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:44627 every 8 connection(s)
I20260928 17:57:15.465577 15873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[15498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618235, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:57:15.475075 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 15873
I20260928 17:57:15.475175 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:57:15.475351 5305 external_mini_cluster.cc:1428] Setting key d99e98fe9bc340152f28c214b7e76ac4
I20260928 17:57:15.475800 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.466407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:44793 (local address 127.5.46.126:41083)
0928 17:57:15.466468 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.466470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.466713 (+ 243us) server_negotiation.cc:407] Connection header received
0928 17:57:15.467269 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.467271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.467285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.467338 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:15.468470 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.469039 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.470142 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.470344 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.470426 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.472797 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:15.472799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:15.472799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:57:15.472807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:15.474232 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.474672 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.474673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.474674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.474731 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.474984 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.474986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.474986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.475151 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:15.475235 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.475608 (+ 373us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.475701 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:57:15.476073 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.466386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:41083 (local address 127.5.46.67:44793)
0928 17:57:15.466580 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.466620 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.467155 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.467436 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.467439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.468146 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:57:15.468351 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.468356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.469172 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.469173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.469997 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.469999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.470434 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.470435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.470569 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.471078 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:57:15.471087 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:57:15.472696 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:57:15.474321 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.474328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.474330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.474597 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.474795 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:57:15.474795 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:57:15.474796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:57:15.474898 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:57:15.475267 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:57:15.475273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:57:15.475413 (+ 140us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.475586 (+ 173us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.475768 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260928 17:57:15.476487 16002 heartbeater.cc:347] Connected to a master server at 127.5.46.126:41083
I20260928 17:57:15.476588 16002 heartbeater.cc:464] Registering TS with master...
I20260928 17:57:15.476737 16002 heartbeater.cc:511] Master 127.5.46.126:41083 requested a full tablet report, sending...
I20260928 17:57:15.477164 15543 ts_manager.cc:194] Registered new tserver with Master: 04d85110aee94e0780c761b458eb286d (127.5.46.67:44627)
I20260928 17:57:15.477604 15543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:44793
I20260928 17:57:15.487453 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:57:15.495313 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.489001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.5.46.126:41083)
0928 17:57:15.489067 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.489069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.489080 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:57:15.489110 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.489111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.489121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.489199 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:57:15.489769 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.490263 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.491029 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.491227 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.491316 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.492649 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:57:15.492651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:57:15.492651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:57:15.492660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:57:15.494070 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.494429 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.494431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.494431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.494500 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:57:15.494705 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:57:15.494706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:57:15.494707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:57:15.494872 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:57:15.494984 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.495119 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.495203 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:57:15.497067 15543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:57:15.498745 15543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:57:15.505373 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.5.46.67:44627)
0928 17:57:15.501381 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.501385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.501580 (+ 195us) server_negotiation.cc:407] Connection header received
0928 17:57:15.501759 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.501762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.501779 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.501834 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:57:15.502126 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.502832 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.504407 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.504927 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.505031 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.505100 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.505243 (+ 143us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.505284 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260928 17:57:15.505373 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:44627 (local address 127.0.0.1:58094)
0928 17:57:15.501464 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.501507 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.501616 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.501951 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.501954 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.501957 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:57:15.502059 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.502062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.502948 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.502949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.504298 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.504301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.505033 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.505034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.505125 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.505139 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.505217 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.505275 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":70,"threads_started":1}
I20260928 17:57:15.506335 15936 tablet_service.cc:1467] Processing CreateTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa (DEFAULT_TABLE table=test-table [id=ae4026c5be234a8ba95a3910d0d9aab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:57:15.506495 16017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41269 (local address 127.0.0.1:54818)
0928 17:57:15.501490 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.501532 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.501600 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.501988 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.501989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.501991 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:57:15.502061 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.502063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.504256 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.504257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.505361 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.505363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.506251 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.506251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.506312 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.506317 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.506384 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.506421 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":90,"threads_started":1}
I20260928 17:57:15.506515 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.5.46.66:41269)
0928 17:57:15.501604 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.501608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.501618 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:57:15.501802 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.501807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.501825 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.501904 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:57:15.502983 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.504148 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.505448 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.506156 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.506233 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.506288 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.506397 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.506438 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":92,"threads_started":1}
I20260928 17:57:15.506767 15936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b838a2a1cb8f40438542fb2eba20c2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:15.507184 15802 tablet_service.cc:1467] Processing CreateTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa (DEFAULT_TABLE table=test-table [id=ae4026c5be234a8ba95a3910d0d9aab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:57:15.507517 15802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b838a2a1cb8f40438542fb2eba20c2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:15.508051 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:34233 (local address 127.0.0.1:42726)
0928 17:57:15.501661 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.501670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.501712 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.502002 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.502003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.502004 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:57:15.502066 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.502067 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.505645 (+ 3578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.505646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.506522 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.506525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.507770 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.507771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.507834 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.507871 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.507939 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.507980 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":32,"thread_start_us":62,"threads_started":1}
I20260928 17:57:15.508090 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.501593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.5.46.65:34233)
0928 17:57:15.501786 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.501791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.501800 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:57:15.501836 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.501838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.501854 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.501924 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:57:15.503930 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.505118 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.506970 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.507674 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.507757 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.507815 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.507957 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.508006 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":87,"threads_started":1}
I20260928 17:57:15.508993 15668 tablet_service.cc:1467] Processing CreateTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa (DEFAULT_TABLE table=test-table [id=ae4026c5be234a8ba95a3910d0d9aab6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:57:15.509402 15668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b838a2a1cb8f40438542fb2eba20c2aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:57:15.510146 16019 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Bootstrap starting.
I20260928 17:57:15.510939 16020 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Bootstrap starting.
I20260928 17:57:15.511080 16019 tablet_bootstrap.cc:654] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:15.511663 16020 tablet_bootstrap.cc:654] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:15.511672 16019 log.cc:824] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:15.512257 16020 log.cc:824] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:15.512528 16019 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: No bootstrap required, opened a new log
I20260928 17:57:15.512604 16019 ts_tablet_manager.cc:1397] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:57:15.512792 16023 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Bootstrap starting.
I20260928 17:57:15.513048 16020 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: No bootstrap required, opened a new log
I20260928 17:57:15.513123 16020 ts_tablet_manager.cc:1397] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:57:15.513919 16023 tablet_bootstrap.cc:654] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Neither blocks nor log segments found. Creating new log.
I20260928 17:57:15.514505 16023 log.cc:824] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Log is configured to *not* fsync() on all Append() calls
I20260928 17:57:15.514530 16020 raft_consensus.cc:348] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.514719 16020 raft_consensus.cc:374] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:15.514780 16020 raft_consensus.cc:729] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9c67b68ebc4a7e97bc3648d53c03f0, State: Initialized, Role: FOLLOWER
I20260928 17:57:15.514920 16020 consensus_queue.cc:260] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.514992 16019 raft_consensus.cc:348] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.515144 16019 raft_consensus.cc:374] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:15.515177 16019 raft_consensus.cc:729] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d85110aee94e0780c761b458eb286d, State: Initialized, Role: FOLLOWER
I20260928 17:57:15.515254 15868 heartbeater.cc:503] Master 127.5.46.126:41083 was elected leader, sending a full tablet report...
I20260928 17:57:15.515303 16019 consensus_queue.cc:260] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.515235 16020 ts_tablet_manager.cc:1428] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:57:15.515597 16019 ts_tablet_manager.cc:1428] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:57:15.516228 16002 heartbeater.cc:503] Master 127.5.46.126:41083 was elected leader, sending a full tablet report...
I20260928 17:57:15.516655 16023 tablet_bootstrap.cc:492] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: No bootstrap required, opened a new log
I20260928 17:57:15.516781 16023 ts_tablet_manager.cc:1397] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 17:57:15.519243 16023 raft_consensus.cc:348] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.519497 16023 raft_consensus.cc:374] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:57:15.519591 16023 raft_consensus.cc:729] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a43b25a9954bdcaeefb6a7010d5eec, State: Initialized, Role: FOLLOWER
I20260928 17:57:15.519776 16023 consensus_queue.cc:260] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.520182 16023 ts_tablet_manager.cc:1428] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:57:15.520254 15734 heartbeater.cc:503] Master 127.5.46.126:41083 was elected leader, sending a full tablet report...
W20260928 17:57:15.555433 15869 tablet.cc:2369] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:57:15.633666 15735 tablet.cc:2369] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:57:15.716161 16003 tablet.cc:2369] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:57:15.724884 16026 raft_consensus.cc:482] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:57:15.725051 16026 raft_consensus.cc:504] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.725368 16026 leader_election.cc:288] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df9c67b68ebc4a7e97bc3648d53c03f0 (127.5.46.66:41269), b7a43b25a9954bdcaeefb6a7010d5eec (127.5.46.65:34233)
I20260928 17:57:15.729661 16018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.725849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58801 (local address 127.5.46.65:34233)
0928 17:57:15.725892 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.725894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.726005 (+ 111us) server_negotiation.cc:407] Connection header received
0928 17:57:15.726049 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.726050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.726062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.726114 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:57:15.726279 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.727059 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.728696 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.729275 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.729362 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.729374 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.729541 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.729581 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:57:15.729655 16016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.725729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:49707 (local address 127.5.46.66:41269)
0928 17:57:15.725782 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.725785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.725798 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:57:15.725871 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.725872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.725885 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.725960 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:57:15.726196 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.727063 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.728696 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.729280 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.729362 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.729374 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.729541 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.729580 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:57:15.729660 16004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.725649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41269 (local address 127.5.46.67:49707)
0928 17:57:15.725719 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.725733 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.725803 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.725979 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.725980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.725982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:57:15.726129 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.726132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.727161 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.727162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.728587 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.728590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.729373 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.729374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.729457 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.729462 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.729517 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.729559 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":67}
I20260928 17:57:15.729665 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.725766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:34233 (local address 127.5.46.67:58801)
0928 17:57:15.725926 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.725937 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:57:15.725987 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:57:15.726125 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:57:15.726126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:57:15.726128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:57:15.726199 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.726201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.727161 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.727162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.728587 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:57:15.728590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.729375 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.729376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:57:15.729457 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.729462 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:57:15.729525 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 17:57:15.729558 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":100,"thread_start_us":100,"threads_started":1}
I20260928 17:57:15.730142 15822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b838a2a1cb8f40438542fb2eba20c2aa" candidate_uuid: "04d85110aee94e0780c761b458eb286d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" is_pre_election: true
I20260928 17:57:15.730176 15688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b838a2a1cb8f40438542fb2eba20c2aa" candidate_uuid: "04d85110aee94e0780c761b458eb286d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" is_pre_election: true
I20260928 17:57:15.730280 15822 raft_consensus.cc:2454] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04d85110aee94e0780c761b458eb286d in term 0.
I20260928 17:57:15.730409 15688 raft_consensus.cc:2454] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04d85110aee94e0780c761b458eb286d in term 0.
I20260928 17:57:15.730567 15892 leader_election.cc:302] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d85110aee94e0780c761b458eb286d, df9c67b68ebc4a7e97bc3648d53c03f0; no voters:
I20260928 17:57:15.730729 16026 raft_consensus.cc:2781] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:57:15.730808 16026 raft_consensus.cc:482] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:57:15.730829 16026 raft_consensus.cc:3037] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:15.731599 16026 raft_consensus.cc:504] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.731710 16026 leader_election.cc:288] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [CANDIDATE]: Term 1 election: Requested vote from peers df9c67b68ebc4a7e97bc3648d53c03f0 (127.5.46.66:41269), b7a43b25a9954bdcaeefb6a7010d5eec (127.5.46.65:34233)
I20260928 17:57:15.731961 15688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b838a2a1cb8f40438542fb2eba20c2aa" candidate_uuid: "04d85110aee94e0780c761b458eb286d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec"
I20260928 17:57:15.732041 15688 raft_consensus.cc:3037] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:15.731978 15822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b838a2a1cb8f40438542fb2eba20c2aa" candidate_uuid: "04d85110aee94e0780c761b458eb286d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0"
I20260928 17:57:15.732101 15822 raft_consensus.cc:3037] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:57:15.732785 15688 raft_consensus.cc:2454] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04d85110aee94e0780c761b458eb286d in term 1.
I20260928 17:57:15.732952 15892 leader_election.cc:302] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04d85110aee94e0780c761b458eb286d, b7a43b25a9954bdcaeefb6a7010d5eec; no voters:
I20260928 17:57:15.733026 15822 raft_consensus.cc:2454] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04d85110aee94e0780c761b458eb286d in term 1.
I20260928 17:57:15.733053 16026 raft_consensus.cc:2781] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:57:15.733201 16026 raft_consensus.cc:686] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 1 LEADER]: Becoming Leader. State: Replica: 04d85110aee94e0780c761b458eb286d, State: Running, Role: LEADER
I20260928 17:57:15.733311 16026 consensus_queue.cc:237] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } }
I20260928 17:57:15.734190 15543 catalog_manager.cc:5495] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d reported cstate change: term changed from 0 to 1, leader changed from <none> to 04d85110aee94e0780c761b458eb286d (127.5.46.67). New cstate: current_term: 1 leader_uuid: "04d85110aee94e0780c761b458eb286d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04d85110aee94e0780c761b458eb286d" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 44627 } health_report { overall_health: HEALTHY } } }
I20260928 17:57:15.740018 16015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.737422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58110 (local address 127.5.46.67:44627)
0928 17:57:15.737484 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 17:57:15.737486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:57:15.737496 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:57:15.737564 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:57:15.737565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:57:15.737582 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:57:15.737647 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:57:15.737776 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.738251 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.739229 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:57:15.739410 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:57:15.739464 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:57:15.739648 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:57:15.739744 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:57:15.739805 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 17:57:15.739908 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 17:57:15.739928 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 17:57:15.741766 15688 raft_consensus.cc:1261] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 1 FOLLOWER]: Refusing update from remote peer 04d85110aee94e0780c761b458eb286d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:57:15.741789 15822 raft_consensus.cc:1261] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 1 FOLLOWER]: Refusing update from remote peer 04d85110aee94e0780c761b458eb286d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:57:15.742144 16029 consensus_queue.cc:1035] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7a43b25a9954bdcaeefb6a7010d5eec" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 34233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:57:15.742282 16026 consensus_queue.cc:1035] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [LEADER]: Connected to new peer: Peer: permanent_uuid: "df9c67b68ebc4a7e97bc3648d53c03f0" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:57:15.745517 16033 mvcc.cc:205] Tried to move back new op lower bound from 7334372293595443200 to 7334372293564350464. Current Snapshot: MvccSnapshot[applied={T|T < 7334372293595443200}]
I20260928 17:57:15.747654 15542 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:57:15.747763 15542 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:57:15.748662 15542 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e547fefbf67440ef8c54f7ce732510d7: Sending DeleteTablet for 3 replicas of tablet b838a2a1cb8f40438542fb2eba20c2aa
I20260928 17:57:15.749068 15668 tablet_service.cc:1514] Processing DeleteTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:57:15 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:42726
I20260928 17:57:15.749068 15802 tablet_service.cc:1514] Processing DeleteTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:57:15 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:54818
I20260928 17:57:15.749127 15936 tablet_service.cc:1514] Processing DeleteTablet for tablet b838a2a1cb8f40438542fb2eba20c2aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:57:15 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:58094
I20260928 17:57:15.749366 16041 tablet_replica.cc:330] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec: stopping tablet replica
I20260928 17:57:15.749488 16041 raft_consensus.cc:2229] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:57:15.749612 16041 raft_consensus.cc:2258] T b838a2a1cb8f40438542fb2eba20c2aa P b7a43b25a9954bdcaeefb6a7010d5eec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:57:15.749711 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15605
I20260928 17:57:15.749771 16042 tablet_replica.cc:330] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: stopping tablet replica
I20260928 17:57:15.749868 16042 raft_consensus.cc:2229] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:57:15.749960 16042 raft_consensus.cc:2258] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:57:15.750429 16042 ts_tablet_manager.cc:1905] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:57:15.752274 16042 ts_tablet_manager.cc:1918] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:57:15.752357 16042 log.cc:1196] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/b838a2a1cb8f40438542fb2eba20c2aa
I20260928 17:57:15.752657 16042 ts_tablet_manager.cc:1939] T b838a2a1cb8f40438542fb2eba20c2aa P df9c67b68ebc4a7e97bc3648d53c03f0: Deleting consensus metadata
I20260928 17:57:15.753050 16043 tablet_replica.cc:330] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: stopping tablet replica
I20260928 17:57:15.753314 16043 raft_consensus.cc:2229] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:57:15.753386 15530 catalog_manager.cc:4841] TS df9c67b68ebc4a7e97bc3648d53c03f0 (127.5.46.66:41269): tablet b838a2a1cb8f40438542fb2eba20c2aa (table test-table [id=ae4026c5be234a8ba95a3910d0d9aab6]) successfully deleted
I20260928 17:57:15.753445 16043 raft_consensus.cc:2258] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:57:15.753906 16043 ts_tablet_manager.cc:1905] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:57:15.755551 16043 ts_tablet_manager.cc:1918] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:57:15.755656 16043 log.cc:1196] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/b838a2a1cb8f40438542fb2eba20c2aa
I20260928 17:57:15.755998 16043 ts_tablet_manager.cc:1939] T b838a2a1cb8f40438542fb2eba20c2aa P 04d85110aee94e0780c761b458eb286d: Deleting consensus metadata
I20260928 17:57:15.756506 15528 catalog_manager.cc:4841] TS 04d85110aee94e0780c761b458eb286d (127.5.46.67:44627): tablet b838a2a1cb8f40438542fb2eba20c2aa (table test-table [id=ae4026c5be234a8ba95a3910d0d9aab6]) successfully deleted
W20260928 17:57:15.757215 15530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:34233 (error 108)
I20260928 17:57:15.757756 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:57:15.757630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:34233 (local address 127.0.0.1:42740)
0928 17:57:15.757683 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 17:57:15.757719 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:34233: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 17:57:15.757784 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15739
W20260928 17:57:15.757886 15530 catalog_manager.cc:4584] TS b7a43b25a9954bdcaeefb6a7010d5eec (127.5.46.65:34233): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b838a2a1cb8f40438542fb2eba20c2aa: Network error: Client connection negotiation failed: client connection to 127.5.46.65:34233: connect: Connection refused (error 111)
I20260928 17:57:15.763381 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15873
I20260928 17:57:15.767880 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 15511
2026-09-28T17:57:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2135 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[16047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[16047](info): set up 2 sockets
Sep 28 17:57:15 dist-test-slave-z2t3 krb5kdc[16047](info): commencing operation
krb5kdc: starting...
W20260928 17:57:16.824990 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:57:16 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:57:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:57:16Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangeradmin_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangeradmin_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangerlookup_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangerlookup_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
I20260928 17:57:17.008428 5305 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskO6aDtj/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/postgres -l logfile start
2026-09-28 17:57:19.873 UTC [16077] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 17:57:19.873 UTC [16077] LOG: listening on IPv4 address "127.5.46.84", port 55559
2026-09-28 17:57:19.878 UTC [16077] LOG: listening on Unix socket "/tmp/.s.PGSQL.55559"
2026-09-28 17:57:19.913 UTC [16080] LOG: database system was shut down at 2026-09-28 17:57:18 UTC
2026-09-28 17:57:19.917 UTC [16077] LOG: database system is ready to accept connections
I20260928 17:57:20.864845 5305 mini_postgres.cc:96] Postgres bound to 55559
2026-09-28 17:57:20.871 UTC [16088] FATAL: database "slave" does not exist
127.5.46.84:55559 - accepting connections
I20260928 17:57:20.871821 5305 mini_ranger.cc:161] Starting Ranger...
I20260928 17:57:20.887187 5305 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 17:57:21.447837 5305 mini_ranger.cc:87] Created ranger Postgres database
I20260928 17:57:21.447986 5305 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin
2026-09-28 17:57:21,750 [I] DB FLAVOR :POSTGRES
2026-09-28 17:57:21,751 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 17:57:21,751 [I] Checking connection..
2026-09-28 17:57:21,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 17:57:21,988 [I] Checking connection passed.
2026-09-28 17:57:21,989 [I] --------- Verifying version history table ---------
2026-09-28 17:57:21,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:57:22,238 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 17:57:22,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:57:22,522 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 17:57:22,523 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 17:57:22,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 17:57:22.769 UTC [16152] WARNING: there is no transaction in progress
2026-09-28 17:57:22,776 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 17:57:22,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:57:23,019 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 17:57:23,020 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 17:57:23,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 17:57:23,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 17:57:23,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z2t3.c.gcp-upstream.internal','N') ;"
2026-09-28 17:57:23,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 17:57:24,006 [I] Table x_portal_user does not exist in database ranger
2026-09-28 17:57:24,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 17:57:24,267 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 17:57:24,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 17:57:24,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 17:57:24,788 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 17:57:24,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 17:57:25.795 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.813 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.830 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.846 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.862 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.880 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.941 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.949 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.964 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.974 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:25.985 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26.000 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26.010 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26.017 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26.025 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26.033 UTC [16260] WARNING: there is no transaction in progress
2026-09-28 17:57:26,722 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 17:57:26,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 17:57:26,946 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 17:57:26,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 17:57:27,210 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 17:57:27,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 17:57:27,464 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 17:57:27,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 17:57:27,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 17:57:27,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z2t3.c.gcp-upstream.internal';"
2026-09-28 17:57:28,196 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 17:57:28,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 17:57:28,475 [I] DB_PATCHES have already been applied
I20260928 17:57:28.480970 5305 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskO6aDtj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 17:57:28.481045 5305 mini_ranger.cc:193] Using host: 127.5.46.84
I20260928 17:57:28.484292 5305 mini_ranger.cc:227] Ranger admin URL: http://127.5.46.84:56693
Sep 28, 2026 5:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 5:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 5:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 5:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 5:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 5:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 5:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 17:57:29.607023 5305 mini_ranger.cc:160] Time spent starting Ranger: real 8.735s user 0.003s sys 0.007s
Sep 28, 2026 5:57:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 5:57:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 28, 2026 5:57:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 5:57:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 5:57:51 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 17:57:51.822193 5305 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangerkms_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/rangerkms_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.46.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.46.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 17:57:51 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618271, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 17:57:51.911577 5305 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 17:57:51.921206 5305 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 17:57:52.929658 5305 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 17:57:52.929764 5305 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms
I20260928 17:57:52.929785 5305 mini_ranger_kms.cc:227] Using postgres at 127.5.46.84:55559
2026-09-28 17:57:53,116 [I] DB FLAVOR :POSTGRES
2026-09-28 17:57:53,117 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 17:57:53,117 [I] Checking connection
2026-09-28 17:57:53,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 17:57:53,367 [I] connection success
2026-09-28 17:57:53,368 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 17:57:53,368 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 17:57:53,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 17:57:53,631 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 17:57:53,631 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 17:57:53,631 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 17:57:53,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:55559/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 17:57:53,891 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 17:57:54.381949 5305 mini_ranger_kms.cc:310] Created kms service
I20260928 17:57:54.565511 5305 mini_ranger_kms.cc:326] Created kudu user
I20260928 17:57:54.613821 5305 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 17:57:54 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618271, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:57:55.039762 5305 mini_ranger_kms.cc:384] Added ranger policy
I20260928 17:57:55.040016 5305 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms:/tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 17:57:55.040062 5305 mini_ranger_kms.cc:240] Using host: 127.5.46.84
I20260928 17:57:55.043795 5305 mini_ranger_kms.cc:276] Ranger KMS PID: 16440
I20260928 17:57:55.043955 5305 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.46.84:52869
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 5:57:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 17:57:57.322610 5305 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.411s user 0.006s sys 0.163s
I20260928 17:57:57.322733 5305 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 17:57:57.322854 5305 mini_ranger_kms.cc:401] 127.5.46.84:52869/kms/v1/keys
Sep 28 17:57:57 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790618277, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 17:57:57 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618277, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
Sep 28 17:57:58 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618277, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
Sep 28 17:58:00 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:58:00.587636 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38513
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:38513
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:52869/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:00.707332 16491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:00.707569 16491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:00.707593 16491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:00.708876 16491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:00.708932 16491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:00.708946 16491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:00.708959 16491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:00.708971 16491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:00.710584 16491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.46.84:52869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:38513
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:00.710839 16491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:00.711102 16491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:00.716362 16497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:00.716385 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:00.716362 16499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:00.716643 16491 server_base.cc:1034] running on GCE node
I20260928 17:58:00.716905 16491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:00.717271 16491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:00.718487 16491 hybrid_clock.cc:648] HybridClock initialized: now 1790618280718470 us; error 37 us; skew 500 ppm
Sep 28 17:58:00 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618280, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:00.727031 16491 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:00.727560 16491 webserver.cc:460] Webserver started at http://127.5.46.126:38719/ using document root <none> and password file <none>
I20260928 17:58:00.727813 16491 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:00.727943 16491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:00.728116 16491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:00 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618280, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:00.791052 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "bcec14247c444c3eb9f0735a74ecba53"
format_stamp: "Formatted at 2026-09-28 17:58:00 on dist-test-slave-z2t3"
server_key: "3842993802344ae2e33272565031c13e"
server_key_iv: "d29d7ca1055770046c21897d8c7b2de0"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:00.791484 16491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "bcec14247c444c3eb9f0735a74ecba53"
format_stamp: "Formatted at 2026-09-28 17:58:00 on dist-test-slave-z2t3"
server_key: "3842993802344ae2e33272565031c13e"
server_key_iv: "d29d7ca1055770046c21897d8c7b2de0"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:00.793054 16491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260928 17:58:00.805397 16506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:00.805614 16491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:58:00.805727 16491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "bcec14247c444c3eb9f0735a74ecba53"
format_stamp: "Formatted at 2026-09-28 17:58:00 on dist-test-slave-z2t3"
server_key: "3842993802344ae2e33272565031c13e"
server_key_iv: "d29d7ca1055770046c21897d8c7b2de0"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:00.805830 16491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:00.825189 16491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:00.827248 16491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:00.827425 16491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:00.832836 16491 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:38513
I20260928 17:58:00.832855 16558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:38513 every 8 connection(s)
I20260928 17:58:00.833316 16491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:00.834120 16559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:00.835726 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 16491
I20260928 17:58:00.835852 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:58:00.836072 5305 external_mini_cluster.cc:1428] Setting key 1268b312281e60c8c918587c7a1beb14
I20260928 17:58:00.837406 16559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: Bootstrap starting.
I20260928 17:58:00.838164 16559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:00.838491 16559 log.cc:824] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:00.839308 16559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: No bootstrap required, opened a new log
I20260928 17:58:00.841907 16559 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } }
I20260928 17:58:00.842092 16559 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:00.842159 16559 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcec14247c444c3eb9f0735a74ecba53, State: Initialized, Role: FOLLOWER
I20260928 17:58:00.842324 16559 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } }
I20260928 17:58:00.842401 16559 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:00.842427 16559 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:00.842491 16559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 17:58:00 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:00.843181 16559 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } }
I20260928 17:58:00.843289 16559 leader_election.cc:302] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcec14247c444c3eb9f0735a74ecba53; no voters:
I20260928 17:58:00.843451 16559 leader_election.cc:288] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:58:00.843509 16564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:00.843734 16564 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [term 1 LEADER]: Becoming Leader. State: Replica: bcec14247c444c3eb9f0735a74ecba53, State: Running, Role: LEADER
I20260928 17:58:00.843803 16559 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:00.843905 16564 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } }
I20260928 17:58:00.844350 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcec14247c444c3eb9f0735a74ecba53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } } }
I20260928 17:58:00.844496 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:00.844790 16564 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcec14247c444c3eb9f0735a74ecba53. Latest consensus state: current_term: 1 leader_uuid: "bcec14247c444c3eb9f0735a74ecba53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcec14247c444c3eb9f0735a74ecba53" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38513 } } }
I20260928 17:58:00.844947 16564 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:00.845115 16573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:00.845274 16559 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 17:58:00.845654 16573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:00.850829 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:00.836628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.5.46.126:38513)
0928 17:58:00.836797 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:00.836807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:00.836855 (+ 48us) server_negotiation.cc:407] Connection header received
0928 17:58:00.837289 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:00.837296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:00.837343 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:00.837504 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:00.838153 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:00.839304 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:00.840129 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:00.840555 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:00.840680 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:00.843207 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:00.843210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:00.843210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:00.843229 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:00.845666 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:00.846104 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:00.846106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:00.846107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:00.846201 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:00.846444 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:00.847686 (+ 1242us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:00.847686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:00.849009 (+ 1323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:00.849850 (+ 841us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:00.849985 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 17:58:00.850458 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260928 17:58:00.857398 16573 catalog_manager.cc:1347] Generated new cluster ID: d3166b6234cb4fc2879ac246b037ebfb
I20260928 17:58:00.857482 16573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:00.873557 16573 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:58:00.874467 16573 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:00.878319 16573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: Generated new TSK 0
I20260928 17:58:00.878643 16573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:58:01.624279 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:52869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:01.716224 16604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:01.716459 16604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:01.716485 16604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:01.717813 16604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:01.717862 16604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:01.717890 16604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:58:01.719506 16604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.46.84:52869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:01.719753 16604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:01.720049 16604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:01.725198 16609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:01.725328 16604 server_base.cc:1034] running on GCE node
W20260928 17:58:01.725450 16612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:01.725458 16610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:01.725832 16604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:01.726117 16604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:01.727319 16604 hybrid_clock.cc:648] HybridClock initialized: now 1790618281727302 us; error 28 us; skew 500 ppm
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:01.730481 16604 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:58:01.730968 16604 webserver.cc:460] Webserver started at http://127.5.46.65:34897/ using document root <none> and password file <none>
I20260928 17:58:01.731266 16604 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:01.731343 16604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:01.731494 16604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:01.746984 16604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "af1d3ba1d2eb49f085130ca601b284b8"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c33a76c5580d457c84eb4dba3b516132"
server_key_iv: "8027c893bfabef6fae78b8609d3cfd08"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:01.747462 16604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "af1d3ba1d2eb49f085130ca601b284b8"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c33a76c5580d457c84eb4dba3b516132"
server_key_iv: "8027c893bfabef6fae78b8609d3cfd08"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:01.750932 16604 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 17:58:01.765965 16619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:01.766301 16604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:01.766479 16604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "af1d3ba1d2eb49f085130ca601b284b8"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c33a76c5580d457c84eb4dba3b516132"
server_key_iv: "8027c893bfabef6fae78b8609d3cfd08"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:01.766680 16604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:01.780068 16604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:01.782722 16604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:01.782958 16604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:01.783280 16604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:01.783705 16604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:01.783805 16604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:01.783936 16604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:01.784014 16604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:01.792747 16604 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:41077
I20260928 17:58:01.792827 16732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:41077 every 8 connection(s)
I20260928 17:58:01.793318 16604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:58:01.800625 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 16604
I20260928 17:58:01.800758 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:58:01.800969 5305 external_mini_cluster.cc:1428] Setting key e9105cef72276f56aec16790117b4b18
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:01.805501 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:01.794209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:60017 (local address 127.5.46.126:38513)
0928 17:58:01.795449 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:01.795451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:01.795460 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:58:01.795487 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:01.795487 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:01.795501 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:01.796407 (+ 906us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:01.796452 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:01.797235 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:01.798556 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:01.798775 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:01.798885 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:01.802017 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:01.802019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:01.802019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:01.802028 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:01.803934 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:01.804421 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:01.804422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:01.804423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:01.804489 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:01.804783 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:01.804784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:01.804785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:01.804916 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:01.804990 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:01.805305 (+ 315us) server_negotiation.cc:299] Negotiation successful
0928 17:58:01.805400 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1206,"thread_start_us":91,"threads_started":1}
I20260928 17:58:01.805718 16735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:01.794147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38513 (local address 127.5.46.65:60017)
0928 17:58:01.794441 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:01.794484 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:01.795105 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:01.795688 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:01.795694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:01.796004 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:01.796304 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:01.796310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:01.797413 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:01.797414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:01.798454 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:01.798457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:01.798894 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:01.798895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:01.799036 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:01.799719 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:01.799728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:01.801886 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:01.804015 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:01.804025 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:01.804028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:01.804322 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:01.804568 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:01.804569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:01.804569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:01.804701 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:01.804992 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:01.804999 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:01.805166 (+ 167us) client_negotiation.cc:769] Sending connection context
0928 17:58:01.805286 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 17:58:01.805446 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":89,"threads_started":1}
I20260928 17:58:01.806419 16733 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38513
I20260928 17:58:01.806543 16733 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:01.806746 16733 heartbeater.cc:511] Master 127.5.46.126:38513 requested a full tablet report, sending...
I20260928 17:58:01.807368 16523 ts_manager.cc:194] Registered new tserver with Master: af1d3ba1d2eb49f085130ca601b284b8 (127.5.46.65:41077)
I20260928 17:58:01.808307 16523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:60017
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:58:01.836019 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:52869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:01.946820 16741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:01.947084 16741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:01.947127 16741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:01.948686 16741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:01.948772 16741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:01.948844 16741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:58:01.950481 16741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.46.84:52869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:01.950788 16741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:01.951042 16741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:01.956703 16749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:01.956830 16741 server_base.cc:1034] running on GCE node
W20260928 17:58:01.957072 16748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:01.956703 16751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:01.957442 16741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:01.957738 16741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:01.958899 16741 hybrid_clock.cc:648] HybridClock initialized: now 1790618281958881 us; error 40 us; skew 500 ppm
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:01.961535 16741 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:58:01.962047 16741 webserver.cc:460] Webserver started at http://127.5.46.66:37907/ using document root <none> and password file <none>
I20260928 17:58:01.962318 16741 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:01.962393 16741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:01.962533 16741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:01.974574 16741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "77a013c840c64a0aadb4f0f2dc83a86c"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c77049c01c4720d2431d1abf989b2da3"
server_key_iv: "0a732ab20891e6712491c72611200db1"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:01.978139 16741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "77a013c840c64a0aadb4f0f2dc83a86c"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c77049c01c4720d2431d1abf989b2da3"
server_key_iv: "0a732ab20891e6712491c72611200db1"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:01.983925 16741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.002s
I20260928 17:58:02.004189 16758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.004474 16741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260928 17:58:02.004578 16741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "77a013c840c64a0aadb4f0f2dc83a86c"
format_stamp: "Formatted at 2026-09-28 17:58:01 on dist-test-slave-z2t3"
server_key: "c77049c01c4720d2431d1abf989b2da3"
server_key_iv: "0a732ab20891e6712491c72611200db1"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:02.004683 16741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 28 17:58:01 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:02.025102 16741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:02.030308 16741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:02.030567 16741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:02.030901 16741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:02.031375 16741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:02.031471 16741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.031549 16741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:02.031610 16741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.038805 16741 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:32927
I20260928 17:58:02.039379 16741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:58:02.044588 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 16741
I20260928 17:58:02.044708 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:58:02.044905 5305 external_mini_cluster.cc:1428] Setting key ed5a63ea366d0af869373095b2b10789
I20260928 17:58:02.049588 16871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:32927 every 8 connection(s)
Sep 28 17:58:02 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:02.062463 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.050688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:33385 (local address 127.5.46.126:38513)
0928 17:58:02.050740 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.050743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.051116 (+ 373us) server_negotiation.cc:407] Connection header received
0928 17:58:02.051829 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.051831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.051865 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.051899 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:02.053249 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.054036 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.055638 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.056002 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.056086 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.058818 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:02.058820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:02.058820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:02.058829 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:02.060706 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.061227 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.061228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.061229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.061288 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.061645 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.061646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.061647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.061795 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:02.061838 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.062263 (+ 425us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.062357 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:58:02.063308 16875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.050370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38513 (local address 127.5.46.66:33385)
0928 17:58:02.050969 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.051026 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.051693 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.052126 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.052130 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.052386 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:02.053126 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.053133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.054160 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.054160 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.055521 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.055524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.056104 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.056105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.056261 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.056923 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:02.056934 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:02.058693 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:02.060811 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:02.060818 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:02.060821 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:02.061134 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:02.061353 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:02.061354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:02.061354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:02.061507 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:02.061880 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:02.061887 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:02.062085 (+ 198us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.062864 (+ 779us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.063055 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":100,"threads_started":1}
I20260928 17:58:02.064050 16872 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38513
I20260928 17:58:02.064148 16872 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:02.064334 16872 heartbeater.cc:511] Master 127.5.46.126:38513 requested a full tablet report, sending...
I20260928 17:58:02.064776 16523 ts_manager.cc:194] Registered new tserver with Master: 77a013c840c64a0aadb4f0f2dc83a86c (127.5.46.66:32927)
I20260928 17:58:02.065258 16523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:33385
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:58:02.093492 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:52869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:02.186484 16878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:02.186744 16878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:02.186789 16878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:02.188220 16878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:02.188293 16878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:02.188341 16878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:58:02.190275 16878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.46.84:52869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:02.190577 16878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:02.190840 16878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:02.196949 16886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:02.196954 16884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:02.197440 16883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:02.198087 16878 server_base.cc:1034] running on GCE node
I20260928 17:58:02.198328 16878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:02.198642 16878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:02.202150 16878 hybrid_clock.cc:648] HybridClock initialized: now 1790618282202097 us; error 30 us; skew 500 ppm
Sep 28 17:58:02 dist-test-slave-z2t3 krb5kdc[16047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618282, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:02.206372 16878 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:58:02.207034 16878 webserver.cc:460] Webserver started at http://127.5.46.67:35419/ using document root <none> and password file <none>
I20260928 17:58:02.207345 16878 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:02.207470 16878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:02.207655 16878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:02 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618282, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:02.221554 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "ed83620b0a6b47619aba0629d80e0ba0"
format_stamp: "Formatted at 2026-09-28 17:58:02 on dist-test-slave-z2t3"
server_key: "ac4a3a764fecfa58aa7b6bf608819323"
server_key_iv: "80a62eb50903ff44e3cca23dfa71f6fd"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:02.225203 16878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "ed83620b0a6b47619aba0629d80e0ba0"
format_stamp: "Formatted at 2026-09-28 17:58:02 on dist-test-slave-z2t3"
server_key: "ac4a3a764fecfa58aa7b6bf608819323"
server_key_iv: "80a62eb50903ff44e3cca23dfa71f6fd"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:02.227332 16878 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260928 17:58:02.254346 16893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.256139 16878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 17:58:02.256287 16878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "ed83620b0a6b47619aba0629d80e0ba0"
format_stamp: "Formatted at 2026-09-28 17:58:02 on dist-test-slave-z2t3"
server_key: "ac4a3a764fecfa58aa7b6bf608819323"
server_key_iv: "80a62eb50903ff44e3cca23dfa71f6fd"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:02.256389 16878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:02.285869 16878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:02.288496 16878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:02.288708 16878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:02.288973 16878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:02.289377 16878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:02.289470 16878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.289539 16878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:02.289593 16878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:02.297788 16878 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:36191
I20260928 17:58:02.298055 17006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:36191 every 8 connection(s)
I20260928 17:58:02.298348 16878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:58:02.303352 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 16878
I20260928 17:58:02.303474 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:58:02.303691 5305 external_mini_cluster.cc:1428] Setting key 8660105c65c6d072805141dc22abb909
Sep 28 17:58:02 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618282, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:02.309350 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.299487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:51139 (local address 127.5.46.126:38513)
0928 17:58:02.299539 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.299542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.299797 (+ 255us) server_negotiation.cc:407] Connection header received
0928 17:58:02.300549 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.300551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.300563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.300599 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:02.301439 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.301954 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.303007 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.303324 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.303395 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.305961 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:02.305963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:02.305963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:02.305973 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:02.307548 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.308109 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.308110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.308111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.308176 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.308489 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.308490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.308490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.308655 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:02.308713 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.309148 (+ 435us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.309251 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:58:02.310111 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.299181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38513 (local address 127.5.46.67:51139)
0928 17:58:02.299634 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.299704 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.300409 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.300805 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.300809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.301082 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:02.301329 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.301335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.302062 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.302063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.302904 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.302906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.303476 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.303478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.303633 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.304288 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:02.304353 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:02.305854 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:02.307659 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:02.307667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:02.307670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:02.308018 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:02.308244 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:02.308245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:02.308246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:02.308394 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:02.308767 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:02.308774 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:02.308981 (+ 207us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.309755 (+ 774us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.309932 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":81,"threads_started":1}
I20260928 17:58:02.310612 17007 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38513
I20260928 17:58:02.310719 17007 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:02.310916 17007 heartbeater.cc:511] Master 127.5.46.126:38513 requested a full tablet report, sending...
I20260928 17:58:02.311318 16523 ts_manager.cc:194] Registered new tserver with Master: ed83620b0a6b47619aba0629d80e0ba0 (127.5.46.67:36191)
I20260928 17:58:02.311790 16523 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:51139
I20260928 17:58:02.315399 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:58:02.325192 16736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.317085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40232 (local address 127.5.46.126:38513)
0928 17:58:02.317127 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.317129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.317927 (+ 798us) server_negotiation.cc:407] Connection header received
0928 17:58:02.318038 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.318039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.318050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.318080 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:02.318877 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.319381 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.320244 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.320512 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.320583 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.322339 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:02.322340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:02.322341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:02.322349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:02.323984 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.324361 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.324362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.324362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.324424 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:02.324635 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:02.324636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:02.324637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:02.324763 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:02.324865 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.325003 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.325082 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:58:02.327113 16523 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:58:02.329020 16523 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:58:02.336367 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.331614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:32927 (local address 127.0.0.1:47618)
0928 17:58:02.332018 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.332030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.332093 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.332553 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.332554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.332557 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:02.332627 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.332628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.333886 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.333887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.334993 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.334996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.336065 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.336066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.336162 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.336178 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.336234 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.336286 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":76,"threads_started":1}
I20260928 17:58:02.336457 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.331667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.5.46.67:36191)
0928 17:58:02.331893 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.331899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.332179 (+ 280us) server_negotiation.cc:407] Connection header received
0928 17:58:02.332227 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.332230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.332248 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.332321 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:02.332647 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.333390 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.335221 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.336000 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.336048 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.336117 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.336328 (+ 211us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.336369 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260928 17:58:02.336838 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.332100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.5.46.66:32927)
0928 17:58:02.332315 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.332320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.332331 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:58:02.332374 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.332435 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.332452 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.332695 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:02.332757 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.333778 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.335132 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.335908 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.336598 (+ 690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.336691 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.336718 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.336761 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260928 17:58:02.336975 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.331570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:36191 (local address 127.0.0.1:45678)
0928 17:58:02.331964 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.332006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.332077 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.332453 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.332457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.332459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:02.332563 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.332565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.333486 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.333487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.335113 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.335114 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.336175 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.336176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.336237 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.336241 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.336860 (+ 619us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.336906 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":38,"thread_start_us":111,"threads_started":1}
I20260928 17:58:02.337671 16806 tablet_service.cc:1467] Processing CreateTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a (DEFAULT_TABLE table=test-table [id=a67249c67f9041909cb485eb2b7715ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:02.337949 16941 tablet_service.cc:1467] Processing CreateTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a (DEFAULT_TABLE table=test-table [id=a67249c67f9041909cb485eb2b7715ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:02.338230 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.331834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41077 (local address 127.0.0.1:58882)
0928 17:58:02.333537 (+ 1703us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.333548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.333594 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.333832 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.333832 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.333834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:02.333867 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.333869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.335074 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.335075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.336202 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.336204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.337985 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.337986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.338048 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.338052 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.338102 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.338143 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1685,"thread_start_us":117,"threads_started":1}
I20260928 17:58:02.338290 16941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68d5db1c54ec49ffb510f0b9e8addb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:02.338452 16806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68d5db1c54ec49ffb510f0b9e8addb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:02.338497 17024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.332282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.5.46.65:41077)
0928 17:58:02.332711 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.332716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.333652 (+ 936us) server_negotiation.cc:407] Connection header received
0928 17:58:02.333688 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.333690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.333705 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.333770 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:02.333937 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.334992 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.336287 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.337821 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.337896 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.337950 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.338391 (+ 441us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.338436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":57,"threads_started":1}
I20260928 17:58:02.339036 16667 tablet_service.cc:1467] Processing CreateTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a (DEFAULT_TABLE table=test-table [id=a67249c67f9041909cb485eb2b7715ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:02.339360 16667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68d5db1c54ec49ffb510f0b9e8addb9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:02.342820 17027 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Bootstrap starting.
I20260928 17:58:02.342919 17025 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Bootstrap starting.
I20260928 17:58:02.343590 17027 tablet_bootstrap.cc:654] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:02.343740 17025 tablet_bootstrap.cc:654] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:02.343943 17027 log.cc:824] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:02.343997 17026 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Bootstrap starting.
I20260928 17:58:02.345269 17026 tablet_bootstrap.cc:654] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:02.345371 17027 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: No bootstrap required, opened a new log
I20260928 17:58:02.345467 17027 ts_tablet_manager.cc:1397] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260928 17:58:02.345602 17026 log.cc:824] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:02.345897 17025 log.cc:824] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:02.346607 17025 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: No bootstrap required, opened a new log
I20260928 17:58:02.346698 17025 ts_tablet_manager.cc:1397] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260928 17:58:02.348026 17027 raft_consensus.cc:348] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.348276 17027 raft_consensus.cc:374] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:02.348212 17025 raft_consensus.cc:348] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.348337 17027 raft_consensus.cc:729] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af1d3ba1d2eb49f085130ca601b284b8, State: Initialized, Role: FOLLOWER
I20260928 17:58:02.348382 17025 raft_consensus.cc:374] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:02.348424 17025 raft_consensus.cc:729] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed83620b0a6b47619aba0629d80e0ba0, State: Initialized, Role: FOLLOWER
I20260928 17:58:02.348462 17027 consensus_queue.cc:260] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.348531 17025 consensus_queue.cc:260] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.348728 17027 ts_tablet_manager.cc:1428] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260928 17:58:02.348798 17025 ts_tablet_manager.cc:1428] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:02.348917 17007 heartbeater.cc:503] Master 127.5.46.126:38513 was elected leader, sending a full tablet report...
I20260928 17:58:02.348973 16733 heartbeater.cc:503] Master 127.5.46.126:38513 was elected leader, sending a full tablet report...
I20260928 17:58:02.349474 17026 tablet_bootstrap.cc:492] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: No bootstrap required, opened a new log
I20260928 17:58:02.349537 17026 ts_tablet_manager.cc:1397] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260928 17:58:02.351099 17026 raft_consensus.cc:348] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.351230 17026 raft_consensus.cc:374] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:02.351251 17026 raft_consensus.cc:729] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a013c840c64a0aadb4f0f2dc83a86c, State: Initialized, Role: FOLLOWER
I20260928 17:58:02.351341 17026 consensus_queue.cc:260] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.351536 17026 ts_tablet_manager.cc:1428] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:02.351663 16872 heartbeater.cc:503] Master 127.5.46.126:38513 was elected leader, sending a full tablet report...
I20260928 17:58:02.472544 17032 raft_consensus.cc:482] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:58:02.472693 17032 raft_consensus.cc:504] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.473129 17032 leader_election.cc:288] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 77a013c840c64a0aadb4f0f2dc83a86c (127.5.46.66:32927), af1d3ba1d2eb49f085130ca601b284b8 (127.5.46.65:41077)
I20260928 17:58:02.477892 17023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.473780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:45139 (local address 127.5.46.66:32927)
0928 17:58:02.473823 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.473826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.473953 (+ 127us) server_negotiation.cc:407] Connection header received
0928 17:58:02.474062 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.474063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.474075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.474104 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:02.474308 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.475034 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.476878 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.477471 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.477517 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.477526 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.477769 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.477814 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 17:58:02.478372 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.473586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:32927 (local address 127.5.46.67:45139)
0928 17:58:02.473892 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.473902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.473987 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.474166 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.474166 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.474167 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:02.474240 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.474242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.475150 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.475151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.476767 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.476770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.477612 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.477613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.477688 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.477692 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.478230 (+ 538us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.478279 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":86,"threads_started":1}
I20260928 17:58:02.478802 17009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.473340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41077 (local address 127.5.46.67:34407)
0928 17:58:02.473407 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.473418 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:02.473463 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:02.473696 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:02.473697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:02.473698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:02.473764 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.473766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.474702 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.474704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.476067 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:02.476070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.478529 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.478531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:02.478608 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.478611 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:58:02.478671 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 17:58:02.478716 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 17:58:02.478896 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68d5db1c54ec49ffb510f0b9e8addb9a" candidate_uuid: "ed83620b0a6b47619aba0629d80e0ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" is_pre_election: true
I20260928 17:58:02.478981 17024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.473433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34407 (local address 127.5.46.65:41077)
0928 17:58:02.473544 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.473546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.473555 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:58:02.473584 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.473585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.473598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.473644 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:02.473838 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.474596 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.476756 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.477352 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.477433 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.477446 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.478841 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.478896 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260928 17:58:02.479095 16826 raft_consensus.cc:2454] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed83620b0a6b47619aba0629d80e0ba0 in term 0.
I20260928 17:58:02.479369 16897 leader_election.cc:302] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77a013c840c64a0aadb4f0f2dc83a86c, ed83620b0a6b47619aba0629d80e0ba0; no voters:
I20260928 17:58:02.479426 16687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68d5db1c54ec49ffb510f0b9e8addb9a" candidate_uuid: "ed83620b0a6b47619aba0629d80e0ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af1d3ba1d2eb49f085130ca601b284b8" is_pre_election: true
I20260928 17:58:02.479540 16687 raft_consensus.cc:2454] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed83620b0a6b47619aba0629d80e0ba0 in term 0.
I20260928 17:58:02.479525 17032 raft_consensus.cc:2781] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:58:02.479568 17032 raft_consensus.cc:482] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:58:02.479589 17032 raft_consensus.cc:3037] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:02.480528 17032 raft_consensus.cc:504] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.480675 17032 leader_election.cc:288] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [CANDIDATE]: Term 1 election: Requested vote from peers 77a013c840c64a0aadb4f0f2dc83a86c (127.5.46.66:32927), af1d3ba1d2eb49f085130ca601b284b8 (127.5.46.65:41077)
I20260928 17:58:02.480870 16687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68d5db1c54ec49ffb510f0b9e8addb9a" candidate_uuid: "ed83620b0a6b47619aba0629d80e0ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af1d3ba1d2eb49f085130ca601b284b8"
I20260928 17:58:02.480959 16687 raft_consensus.cc:3037] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:02.481058 16826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68d5db1c54ec49ffb510f0b9e8addb9a" candidate_uuid: "ed83620b0a6b47619aba0629d80e0ba0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77a013c840c64a0aadb4f0f2dc83a86c"
I20260928 17:58:02.481140 16826 raft_consensus.cc:3037] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:02.481928 16687 raft_consensus.cc:2454] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed83620b0a6b47619aba0629d80e0ba0 in term 1.
I20260928 17:58:02.481993 16826 raft_consensus.cc:2454] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed83620b0a6b47619aba0629d80e0ba0 in term 1.
I20260928 17:58:02.482223 16895 leader_election.cc:302] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af1d3ba1d2eb49f085130ca601b284b8, ed83620b0a6b47619aba0629d80e0ba0; no voters:
I20260928 17:58:02.482343 17032 raft_consensus.cc:2781] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:02.482563 17032 raft_consensus.cc:686] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 1 LEADER]: Becoming Leader. State: Replica: ed83620b0a6b47619aba0629d80e0ba0, State: Running, Role: LEADER
I20260928 17:58:02.482683 17032 consensus_queue.cc:237] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } }
I20260928 17:58:02.483616 16523 catalog_manager.cc:5495] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed83620b0a6b47619aba0629d80e0ba0 (127.5.46.67). New cstate: current_term: 1 leader_uuid: "ed83620b0a6b47619aba0629d80e0ba0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed83620b0a6b47619aba0629d80e0ba0" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 36191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 } health_report { overall_health: UNKNOWN } } }
Sep 28 17:58:02 dist-test-slave-z2t3 krb5kdc[16047](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:02.528438 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.525209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45686 (local address 127.5.46.67:36191)
0928 17:58:02.525260 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:02.525263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:02.525275 (+ 12us) server_negotiation.cc:407] Connection header received
0928 17:58:02.525369 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:02.525370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:02.525395 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:02.525493 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:58:02.525718 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.526259 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.527373 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:02.527669 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:02.527760 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:02.527983 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:58:02.528130 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:58:02.528233 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:02.528321 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 17:58:02.528348 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:58:02.536901 16687 raft_consensus.cc:1261] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 1 FOLLOWER]: Refusing update from remote peer ed83620b0a6b47619aba0629d80e0ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:02.537406 17032 consensus_queue.cc:1035] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af1d3ba1d2eb49f085130ca601b284b8" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:02.540292 16826 raft_consensus.cc:1261] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 1 FOLLOWER]: Refusing update from remote peer ed83620b0a6b47619aba0629d80e0ba0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:02.540757 17032 consensus_queue.cc:1035] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77a013c840c64a0aadb4f0f2dc83a86c" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 32927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 17:58:02.546023 16734 tablet.cc:2369] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:58:02.546958 17038 mvcc.cc:205] Tried to move back new op lower bound from 7334372485267136512 to 7334372485049688064. Current Snapshot: MvccSnapshot[applied={T|T < 7334372485267136512}]
I20260928 17:58:02.547300 17041 mvcc.cc:205] Tried to move back new op lower bound from 7334372485267136512 to 7334372485049688064. Current Snapshot: MvccSnapshot[applied={T|T < 7334372485267136512}]
I20260928 17:58:02.550042 17039 mvcc.cc:205] Tried to move back new op lower bound from 7334372485267136512 to 7334372485049688064. Current Snapshot: MvccSnapshot[applied={T|T < 7334372485267136512}]
I20260928 17:58:02.554399 16523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:02.554519 16523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:02.555524 16523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcec14247c444c3eb9f0735a74ecba53: Sending DeleteTablet for 3 replicas of tablet 68d5db1c54ec49ffb510f0b9e8addb9a
I20260928 17:58:02.556111 16806 tablet_service.cc:1514] Processing DeleteTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:02 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:47618
I20260928 17:58:02.556361 16941 tablet_service.cc:1514] Processing DeleteTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:02 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:45678
I20260928 17:58:02.556756 16667 tablet_service.cc:1514] Processing DeleteTablet for tablet 68d5db1c54ec49ffb510f0b9e8addb9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:02 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:58882
I20260928 17:58:02.557344 17049 tablet_replica.cc:330] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8: stopping tablet replica
I20260928 17:58:02.557518 17049 raft_consensus.cc:2229] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:02.557740 17049 raft_consensus.cc:2258] T 68d5db1c54ec49ffb510f0b9e8addb9a P af1d3ba1d2eb49f085130ca601b284b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:02.557911 17047 tablet_replica.cc:330] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: stopping tablet replica
I20260928 17:58:02.558063 17048 tablet_replica.cc:330] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: stopping tablet replica
I20260928 17:58:02.558298 17048 raft_consensus.cc:2229] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:58:02.558339 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 16604
I20260928 17:58:02.558457 17047 raft_consensus.cc:2229] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:02.558557 17047 raft_consensus.cc:2258] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:02.560897 17048 raft_consensus.cc:2258] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:02.561712 17047 ts_tablet_manager.cc:1905] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:02.564363 17048 ts_tablet_manager.cc:1905] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:02.567925 17047 ts_tablet_manager.cc:1918] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:02.569224 17048 ts_tablet_manager.cc:1918] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:02.569411 17048 log.cc:1196] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/68d5db1c54ec49ffb510f0b9e8addb9a
I20260928 17:58:02.569829 17047 log.cc:1196] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/68d5db1c54ec49ffb510f0b9e8addb9a
I20260928 17:58:02.569860 17048 ts_tablet_manager.cc:1939] T 68d5db1c54ec49ffb510f0b9e8addb9a P ed83620b0a6b47619aba0629d80e0ba0: Deleting consensus metadata
I20260928 17:58:02.570129 17047 ts_tablet_manager.cc:1939] T 68d5db1c54ec49ffb510f0b9e8addb9a P 77a013c840c64a0aadb4f0f2dc83a86c: Deleting consensus metadata
I20260928 17:58:02.570580 16508 catalog_manager.cc:4841] TS ed83620b0a6b47619aba0629d80e0ba0 (127.5.46.67:36191): tablet 68d5db1c54ec49ffb510f0b9e8addb9a (table test-table [id=a67249c67f9041909cb485eb2b7715ed]) successfully deleted
I20260928 17:58:02.570673 16510 catalog_manager.cc:4841] TS 77a013c840c64a0aadb4f0f2dc83a86c (127.5.46.66:32927): tablet 68d5db1c54ec49ffb510f0b9e8addb9a (table test-table [id=a67249c67f9041909cb485eb2b7715ed]) successfully deleted
W20260928 17:58:02.572170 16508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:41077 (error 108)
I20260928 17:58:02.578266 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:02.578076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41077 (local address 127.0.0.1:58890)
0928 17:58:02.578144 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:02.578211 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 17:58:02.578486 16508 catalog_manager.cc:4584] TS af1d3ba1d2eb49f085130ca601b284b8 (127.5.46.65:41077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 68d5db1c54ec49ffb510f0b9e8addb9a: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41077: connect: Connection refused (error 111)
I20260928 17:58:02.580024 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 16741
I20260928 17:58:02.587383 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 16878
I20260928 17:58:02.592581 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 16491
I20260928 17:58:02.597453 5305 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 17:58:02.942338 5305 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 17:58:02.942463 5305 mini_ranger.cc:66] Stopping Ranger...
I20260928 17:58:03.287415 5305 mini_ranger.cc:68] Stopped Ranger
2026-09-28 17:58:03.287 UTC [16077] LOG: received smart shutdown request
2026-09-28 17:58:03.291 UTC [16077] LOG: background worker "logical replication launcher" (PID 16086) exited with exit code 1
2026-09-28 17:58:03.291 UTC [16081] LOG: shutting down
2026-09-28 17:58:03.497 UTC [16077] LOG: database system is shut down
2026-09-28T17:58:03Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47834 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:58:03 dist-test-slave-z2t3 krb5kdc[17059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:58:03 dist-test-slave-z2t3 krb5kdc[17059](info): set up 2 sockets
Sep 28 17:58:03 dist-test-slave-z2t3 krb5kdc[17059](info): commencing operation
krb5kdc: starting...
W20260928 17:58:04.668051 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:58:04 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:58:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangeradmin_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangeradmin_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangerlookup_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangerlookup_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
I20260928 17:58:04.850158 5305 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskO6aDtj/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/postgres -l logfile start
2026-09-28 17:58:07.736 UTC [17089] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 17:58:07.736 UTC [17089] LOG: listening on IPv4 address "127.5.46.84", port 45289
2026-09-28 17:58:07.741 UTC [17089] LOG: listening on Unix socket "/tmp/.s.PGSQL.45289"
2026-09-28 17:58:07.765 UTC [17092] LOG: database system was shut down at 2026-09-28 17:58:05 UTC
2026-09-28 17:58:07.769 UTC [17089] LOG: database system is ready to accept connections
I20260928 17:58:08.727725 5305 mini_postgres.cc:96] Postgres bound to 45289
2026-09-28 17:58:08.733 UTC [17100] FATAL: database "slave" does not exist
127.5.46.84:45289 - accepting connections
I20260928 17:58:08.734257 5305 mini_ranger.cc:161] Starting Ranger...
I20260928 17:58:08.749418 5305 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 17:58:09.339040 5305 mini_ranger.cc:87] Created ranger Postgres database
I20260928 17:58:09.339135 5305 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin
2026-09-28 17:58:09,627 [I] DB FLAVOR :POSTGRES
2026-09-28 17:58:09,627 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 17:58:09,627 [I] Checking connection..
2026-09-28 17:58:09,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 17:58:09,880 [I] Checking connection passed.
2026-09-28 17:58:09,880 [I] --------- Verifying version history table ---------
2026-09-28 17:58:09,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:58:10,142 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 17:58:10,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:58:10,409 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 17:58:10,410 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 17:58:10,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 17:58:10.663 UTC [17164] WARNING: there is no transaction in progress
2026-09-28 17:58:10,670 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 17:58:10,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 17:58:10,927 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 17:58:10,927 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 17:58:10,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 17:58:11,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 17:58:11,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-z2t3.c.gcp-upstream.internal','N') ;"
2026-09-28 17:58:11,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 17:58:11,947 [I] Table x_portal_user does not exist in database ranger
2026-09-28 17:58:11,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 17:58:12,200 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 17:58:12,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 17:58:12,448 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 17:58:12,689 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 17:58:12,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 17:58:13.700 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.717 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.735 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.753 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.770 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.790 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.850 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.857 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.873 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.884 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.896 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.912 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.923 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.932 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.940 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:13.947 UTC [17272] WARNING: there is no transaction in progress
2026-09-28 17:58:14,689 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 17:58:14,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 17:58:14,949 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 17:58:14,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 17:58:15,187 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 17:58:15,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 17:58:15,425 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 17:58:15,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 17:58:15,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 17:58:15,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-z2t3.c.gcp-upstream.internal';"
2026-09-28 17:58:16,118 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 17:58:16,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 17:58:16,350 [I] DB_PATCHES have already been applied
I20260928 17:58:16.356529 5305 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskO6aDtj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 17:58:16.356632 5305 mini_ranger.cc:193] Using host: 127.5.46.84
I20260928 17:58:16.359678 5305 mini_ranger.cc:227] Ranger admin URL: http://127.5.46.84:56791
Sep 28, 2026 5:58:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 5:58:16 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 5:58:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 5:58:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 5:58:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 5:58:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 5:58:17 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 17:58:17.491032 5305 mini_ranger.cc:160] Time spent starting Ranger: real 8.757s user 0.003s sys 0.006s
Sep 28, 2026 5:58:37 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 5:58:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 5:58:37 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 5:58:37 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 5:58:38 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 5:58:38 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 17:58:39.489167 5305 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.46.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.46.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangerkms_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.46.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/rangerkms_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.46.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.46.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.46.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/HTTP_127.5.46.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 17:58:39 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618319, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 17:58:39.600915 5305 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 17:58:39.611609 5305 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 17:58:40.636245 5305 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 17:58:40.636343 5305 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms
I20260928 17:58:40.636366 5305 mini_ranger_kms.cc:227] Using postgres at 127.5.46.84:45289
2026-09-28 17:58:40,810 [I] DB FLAVOR :POSTGRES
2026-09-28 17:58:40,810 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 17:58:40,811 [I] Checking connection
2026-09-28 17:58:40,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 17:58:41,057 [I] connection success
2026-09-28 17:58:41,057 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 17:58:41,057 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 17:58:41,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 17:58:41,306 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 17:58:41,307 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 17:58:41,307 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 17:58:41,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.46.84:45289/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 17:58:41,584 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 17:58:42.051976 5305 mini_ranger_kms.cc:310] Created kms service
I20260928 17:58:42.226054 5305 mini_ranger_kms.cc:326] Created kudu user
I20260928 17:58:42.270045 5305 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 17:58:42 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618319, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:42.670914 5305 mini_ranger_kms.cc:384] Added ranger policy
I20260928 17:58:42.671146 5305 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms:/tmp/dist-test-taskO6aDtj/build/release/bin/postgresql.jar:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskO6aDtj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 17:58:42.671171 5305 mini_ranger_kms.cc:240] Using host: 127.5.46.84
I20260928 17:58:42.674551 5305 mini_ranger_kms.cc:276] Ranger KMS PID: 17452
I20260928 17:58:42.674656 5305 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.46.84:43945
Sep 28, 2026 5:58:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 5:58:43 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 5:58:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 5:58:43 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 5:58:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 5:58:44 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 17:58:44.944217 5305 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.343s user 0.009s sys 0.143s
I20260928 17:58:44.944337 5305 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 17:58:44.944391 5305 mini_ranger_kms.cc:401] 127.5.46.84:43945/kms/v1/keys
Sep 28 17:58:44 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790618324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 17:58:45 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
Sep 28 17:58:45 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.46.84@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
Sep 28 17:58:48 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:58:48.099793 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44301
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:44301
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:48.182092 17503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:48.182329 17503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:48.182353 17503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:48.183450 17503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:48.183486 17503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:48.183501 17503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:48.183513 17503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:48.183525 17503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:48.185031 17503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.46.84:43945/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:44301
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:48.185228 17503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:48.185406 17503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:48.190857 17509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:48.190867 17508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:48.190863 17511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:48.190881 17503 server_base.cc:1034] running on GCE node
I20260928 17:58:48.191313 17503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:48.191612 17503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:48.192826 17503 hybrid_clock.cc:648] HybridClock initialized: now 1790618328192781 us; error 43 us; skew 500 ppm
Sep 28 17:58:48 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618328, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:48.195545 17503 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:48.196071 17503 webserver.cc:460] Webserver started at http://127.5.46.126:34931/ using document root <none> and password file <none>
I20260928 17:58:48.196296 17503 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:48.196357 17503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:48.196513 17503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:48 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618328, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:48.243413 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "eab64796e52449699baaa8420e70d004"
format_stamp: "Formatted at 2026-09-28 17:58:48 on dist-test-slave-z2t3"
server_key: "0c9d5c07371ce14a5717d4db146e5393"
server_key_iv: "f7a36a5a34730f9af5ffdd144c05749f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:48.243908 17503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "eab64796e52449699baaa8420e70d004"
format_stamp: "Formatted at 2026-09-28 17:58:48 on dist-test-slave-z2t3"
server_key: "0c9d5c07371ce14a5717d4db146e5393"
server_key_iv: "f7a36a5a34730f9af5ffdd144c05749f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:48.245491 17503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:58:48.257736 17518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:48.257977 17503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:48.258088 17503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "eab64796e52449699baaa8420e70d004"
format_stamp: "Formatted at 2026-09-28 17:58:48 on dist-test-slave-z2t3"
server_key: "0c9d5c07371ce14a5717d4db146e5393"
server_key_iv: "f7a36a5a34730f9af5ffdd144c05749f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:48.258210 17503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:48.267354 17503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:48.269546 17503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:48.269726 17503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:48.274613 17503 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:44301
I20260928 17:58:48.274685 17570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:44301 every 8 connection(s)
I20260928 17:58:48.275035 17503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:48.275588 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 17503
I20260928 17:58:48.275691 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:58:48.275941 5305 external_mini_cluster.cc:1428] Setting key 26b7762d1d36cb607d3dfef13e4479b9
I20260928 17:58:48.275947 17571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:48.279350 17571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Bootstrap starting.
I20260928 17:58:48.280048 17571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:48.280346 17571 log.cc:824] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:48.281046 17571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: No bootstrap required, opened a new log
I20260928 17:58:48.282860 17571 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:48.283032 17571 raft_consensus.cc:374] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:48.283097 17571 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab64796e52449699baaa8420e70d004, State: Initialized, Role: FOLLOWER
I20260928 17:58:48.283238 17571 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:48.283340 17571 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:48.283394 17571 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:48.283466 17571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:48.284149 17571 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:48.284272 17571 leader_election.cc:302] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab64796e52449699baaa8420e70d004; no voters:
I20260928 17:58:48.284459 17571 leader_election.cc:288] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:58:48.284484 17576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Leader election won for term 1
Sep 28 17:58:48 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:48.284744 17576 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 LEADER]: Becoming Leader. State: Replica: eab64796e52449699baaa8420e70d004, State: Running, Role: LEADER
I20260928 17:58:48.284837 17571 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:48.284914 17576 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:48.285319 17576 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eab64796e52449699baaa8420e70d004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } } }
I20260928 17:58:48.285485 17576 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:48.285669 17576 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab64796e52449699baaa8420e70d004. Latest consensus state: current_term: 1 leader_uuid: "eab64796e52449699baaa8420e70d004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } } }
I20260928 17:58:48.285775 17576 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:48.285950 17585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:48.286288 17571 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 17:58:48.286656 17585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:48.288969 17593 mvcc.cc:205] Tried to move back new op lower bound from 7334372672665661440 to 7334372672655523840. Current Snapshot: MvccSnapshot[applied={T|T < 7334372672665661440}]
I20260928 17:58:48.289460 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:48.276498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60290 (local address 127.5.46.126:44301)
0928 17:58:48.276703 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:48.276715 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:48.276784 (+ 69us) server_negotiation.cc:407] Connection header received
0928 17:58:48.277609 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:48.277619 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:48.277690 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:48.277863 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:48.278701 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:48.280138 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:48.281274 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:48.281898 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:48.282006 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:48.285228 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:48.285232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:48.285232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:48.285247 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:48.287313 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:48.287757 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:48.287760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:48.287761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:48.287893 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:48.288146 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:48.288147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:48.288148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:48.288479 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:48.288660 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:48.289129 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 17:58:48.289256 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260928 17:58:48.290166 17585 catalog_manager.cc:1347] Generated new cluster ID: 805daae4e8b9411c8abfe64c4867f677
I20260928 17:58:48.290256 17585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:48.309569 17585 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:58:48.310516 17585 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:48.316036 17585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Generated new TSK 0
I20260928 17:58:48.316318 17585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:58:49.006726 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:49.095553 17616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:49.095800 17616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:49.095842 17616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:49.097041 17616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:49.097113 17616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:49.097159 17616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:58:49.098783 17616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:49.099051 17616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:49.099370 17616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:49.104571 17621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.104713 17624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.104571 17622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.105582 17616 server_base.cc:1034] running on GCE node
I20260928 17:58:49.105822 17616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:49.106077 17616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:49.107251 17616 hybrid_clock.cc:648] HybridClock initialized: now 1790618329107232 us; error 41 us; skew 500 ppm
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:49.109891 17616 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:58:49.110394 17616 webserver.cc:460] Webserver started at http://127.5.46.65:45265/ using document root <none> and password file <none>
I20260928 17:58:49.110656 17616 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:49.110733 17616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:49.110873 17616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:49.124055 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "d0f1898458ef47cf8dd81f19a1299e6d"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "b838b05466239af37cdc80c23fa66766"
server_key_iv: "b6f9287684f334121ae8b253f270a27d"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.124559 17616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "d0f1898458ef47cf8dd81f19a1299e6d"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "b838b05466239af37cdc80c23fa66766"
server_key_iv: "b6f9287684f334121ae8b253f270a27d"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.126560 17616 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260928 17:58:49.143436 17631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.143746 17616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:49.143931 17616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "d0f1898458ef47cf8dd81f19a1299e6d"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "b838b05466239af37cdc80c23fa66766"
server_key_iv: "b6f9287684f334121ae8b253f270a27d"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.144119 17616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:49.157923 17616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:49.161363 17616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:49.161543 17616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:49.161895 17616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:49.162333 17616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:49.162384 17616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.162418 17616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:49.162443 17616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.169936 17616 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33609
I20260928 17:58:49.170015 17744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33609 every 8 connection(s)
I20260928 17:58:49.170433 17616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:58:49.174973 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 17616
I20260928 17:58:49.175119 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:58:49.175310 5305 external_mini_cluster.cc:1428] Setting key 92129a7e4c09b0d956f6aae8158c4d4c
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:49.182521 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.171392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:42491 (local address 127.5.46.126:44301)
0928 17:58:49.171991 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:49.171993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:49.172006 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:58:49.172388 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:49.172389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:49.172399 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:49.172473 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:49.173196 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.173715 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.175217 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.175539 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.175601 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.178296 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:49.178298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:49.178298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:49.178307 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:49.180482 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.181071 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.181072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.181073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.181148 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.181579 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.181580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.181580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.181745 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:49.181826 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:49.182291 (+ 465us) server_negotiation.cc:299] Negotiation successful
0928 17:58:49.182401 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":74,"threads_started":1}
I20260928 17:58:49.183082 17747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.171301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.65:42491)
0928 17:58:49.171520 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:49.171572 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:49.172273 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:49.172551 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:49.172556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:49.172795 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:49.173080 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.173086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.173818 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.173819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.175113 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.175116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.175683 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.175684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.175825 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.176433 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:49.176445 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:49.178181 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:49.180600 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.180608 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.180612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.180939 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.181240 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.181241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.181241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.181475 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.181838 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:49.181846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:49.182084 (+ 238us) client_negotiation.cc:769] Sending connection context
0928 17:58:49.182229 (+ 145us) client_negotiation.cc:240] Negotiation successful
0928 17:58:49.182443 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260928 17:58:49.183913 17745 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:49.184031 17745 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:49.184290 17745 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:49.184958 17535 ts_manager.cc:194] Registered new tserver with Master: d0f1898458ef47cf8dd81f19a1299e6d (127.5.46.65:33609)
I20260928 17:58:49.185950 17535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:42491
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:58:49.208930 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:49.311071 17753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:49.311378 17753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:49.311429 17753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:49.313293 17753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:49.313357 17753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:49.313406 17753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:58:49.315704 17753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:49.316032 17753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:49.316310 17753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:49.322829 17763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.323159 17761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.324460 17753 server_base.cc:1034] running on GCE node
W20260928 17:58:49.324671 17760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.324918 17753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:49.325222 17753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:49.326377 17753 hybrid_clock.cc:648] HybridClock initialized: now 1790618329326368 us; error 31 us; skew 500 ppm
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:49.329385 17753 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:58:49.329941 17753 webserver.cc:460] Webserver started at http://127.5.46.66:42657/ using document root <none> and password file <none>
I20260928 17:58:49.330219 17753 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:49.330289 17753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:49.330433 17753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:49.351339 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "b8c8ab10b5224c09a1aa3a9202115034"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "8c3c7fc0fc9aa803b9fc66d1d369b464"
server_key_iv: "0c1992ba0ea4d0509fb999d34ad3f677"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.351945 17753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "b8c8ab10b5224c09a1aa3a9202115034"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "8c3c7fc0fc9aa803b9fc66d1d369b464"
server_key_iv: "0c1992ba0ea4d0509fb999d34ad3f677"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.357187 17753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260928 17:58:49.372380 17770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.372655 17753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:58:49.372749 17753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "b8c8ab10b5224c09a1aa3a9202115034"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "8c3c7fc0fc9aa803b9fc66d1d369b464"
server_key_iv: "0c1992ba0ea4d0509fb999d34ad3f677"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.372845 17753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:49.389272 17753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:49.392228 17753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:49.392375 17753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:49.392598 17753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:49.392949 17753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:49.392983 17753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.393013 17753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:49.393038 17753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.415149 17753 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:34243
I20260928 17:58:49.415244 17883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:34243 every 8 connection(s)
I20260928 17:58:49.415776 17753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:58:49.418401 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 17753
I20260928 17:58:49.418510 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:58:49.418708 5305 external_mini_cluster.cc:1428] Setting key a61655ead6b0822993d64cfbf9439e4e
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:49.429108 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.417082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.66:34883)
0928 17:58:49.417374 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:49.417422 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:49.418118 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:49.418566 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:49.418571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:49.419064 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:49.419448 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.419456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.420476 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.420477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.421957 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.421960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.422552 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.422553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.422695 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.423384 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:49.423395 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:49.425111 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:49.427235 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.427244 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.427247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.427600 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.427831 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.427832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.427832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.428002 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.428331 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:49.428338 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:49.428533 (+ 195us) client_negotiation.cc:769] Sending connection context
0928 17:58:49.428669 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 17:58:49.428848 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":87,"threads_started":1}
I20260928 17:58:49.429601 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.417380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:34883 (local address 127.5.46.126:44301)
0928 17:58:49.417431 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:49.417433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:49.417547 (+ 114us) server_negotiation.cc:407] Connection header received
0928 17:58:49.418299 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:49.418301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:49.418315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:49.418352 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:49.419555 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.420359 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.422067 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.422384 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.422434 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.425237 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:49.425239 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:49.425239 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:49.425248 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:49.427119 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.427693 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.427694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.427695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.427760 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.428083 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.428084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.428084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.428235 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:49.428302 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:49.429393 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0928 17:58:49.429495 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:58:49.429960 17884 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:49.430085 17884 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:49.430297 17884 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:49.430730 17535 ts_manager.cc:194] Registered new tserver with Master: b8c8ab10b5224c09a1aa3a9202115034 (127.5.46.66:34243)
I20260928 17:58:49.431224 17535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:34883
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:58:49.455192 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:49.574129 17889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:49.574450 17889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:49.574491 17889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:49.576342 17889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:49.576411 17889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:49.576457 17889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:58:49.578918 17889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:49.579219 17889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:49.579505 17889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:49.587126 17898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.587177 17896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.587199 17895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.587936 17889 server_base.cc:1034] running on GCE node
I20260928 17:58:49.588223 17889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:49.588579 17889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:49.589792 17889 hybrid_clock.cc:648] HybridClock initialized: now 1790618329589769 us; error 40 us; skew 500 ppm
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:49.593544 17889 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:58:49.594129 17889 webserver.cc:460] Webserver started at http://127.5.46.67:37837/ using document root <none> and password file <none>
I20260928 17:58:49.594427 17889 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:49.594491 17889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:49.594633 17889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:49.610054 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "a9c89157a05842b0b9dd10cc07f89aee"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "4bb20d17635c217f3006a24797331b4f"
server_key_iv: "a7ea29c27712d8c1d49748710ce0a57f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.610481 17889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "a9c89157a05842b0b9dd10cc07f89aee"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "4bb20d17635c217f3006a24797331b4f"
server_key_iv: "a7ea29c27712d8c1d49748710ce0a57f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.613853 17889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260928 17:58:49.636852 17905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.637080 17889 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260928 17:58:49.637180 17889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "a9c89157a05842b0b9dd10cc07f89aee"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "4bb20d17635c217f3006a24797331b4f"
server_key_iv: "a7ea29c27712d8c1d49748710ce0a57f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.637286 17889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:49.665288 17889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:49.667886 17889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:49.668033 17889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:49.668283 17889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:49.668617 17889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:49.668666 17889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.668699 17889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:49.668715 17889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.676916 17889 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:41131
I20260928 17:58:49.677008 18018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:41131 every 8 connection(s)
I20260928 17:58:49.677374 17889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:49.684005 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 17889
I20260928 17:58:49.684127 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:58:49.684321 5305 external_mini_cluster.cc:1428] Setting key 6198273d49760b551a2c886dbd193165
I20260928 17:58:49.687312 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.678178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:43451 (local address 127.5.46.126:44301)
0928 17:58:49.678238 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:49.678240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:49.678463 (+ 223us) server_negotiation.cc:407] Connection header received
0928 17:58:49.679001 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:49.679003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:49.679015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:49.679074 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:49.680022 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.680529 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.681530 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.681791 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.681884 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.684144 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:49.684145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:49.684146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:49.684155 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:49.685898 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.686319 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.686320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.686321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.686388 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:49.686620 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:49.686621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:49.686621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:49.686766 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:49.686833 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:49.687122 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 17:58:49.687213 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:58:49.687404 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:49.678183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.67:43451)
0928 17:58:49.678339 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:49.678385 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:49.678888 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:49.679162 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:49.679169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:49.679651 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:49.679921 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.679927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.680628 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.680630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.681432 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:49.681434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:49.681911 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:49.681912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:49.682036 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:49.682615 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:49.682623 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:49.684014 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:49.685981 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.685987 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.685989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.686245 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.686454 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:49.686454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:49.686455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:49.686563 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:49.686841 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:49.686845 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:49.686986 (+ 141us) client_negotiation.cc:769] Sending connection context
0928 17:58:49.687072 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 17:58:49.687208 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260928 17:58:49.688283 18019 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:49.688385 18019 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:49.688570 18019 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:49.688956 17535 ts_manager.cc:194] Registered new tserver with Master: a9c89157a05842b0b9dd10cc07f89aee (127.5.46.67:41131)
I20260928 17:58:49.689333 17535 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:43451
I20260928 17:58:49.703317 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:58:49.704107 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 17616
I20260928 17:58:49.710054 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 17753
I20260928 17:58:49.715171 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 17889
I20260928 17:58:49.720163 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 17503
I20260928 17:58:49.724763 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44301
--webserver_interface=127.5.46.126
--webserver_port=34931
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:44301
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:49.806850 18022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:49.807075 18022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:49.807098 18022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:49.808224 18022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:49.808261 18022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:49.808276 18022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:49.808288 18022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:49.808300 18022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:49.809932 18022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:44301
--ranger_config_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=34931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:49.810133 18022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:49.810305 18022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:49.815270 18028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.815442 18022 server_base.cc:1034] running on GCE node
W20260928 17:58:49.815272 18030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:49.815335 18027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:49.815889 18022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:49.816190 18022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:49.817369 18022 hybrid_clock.cc:648] HybridClock initialized: now 1790618329817359 us; error 30 us; skew 500 ppm
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:49.819957 18022 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:49.820443 18022 webserver.cc:460] Webserver started at http://127.5.46.126:34931/ using document root <none> and password file <none>
I20260928 17:58:49.820670 18022 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:49.820727 18022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 17:58:49 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618329, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:49.878253 18022 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:49.879101 18038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:49.879290 18022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:49.879421 18022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "eab64796e52449699baaa8420e70d004"
format_stamp: "Formatted at 2026-09-28 17:58:48 on dist-test-slave-z2t3"
server_key: "0c9d5c07371ce14a5717d4db146e5393"
server_key_iv: "f7a36a5a34730f9af5ffdd144c05749f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:49.879817 18022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:49.895318 18022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:49.897564 18022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:49.897691 18022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:49.902218 18022 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:44301
I20260928 17:58:49.902374 18090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:44301 every 8 connection(s)
I20260928 17:58:49.902700 18022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:49.904183 18091 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:58:49.904858 18091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Bootstrap starting.
I20260928 17:58:49.910786 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18022
I20260928 17:58:49.911113 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:33609
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:44301
--webserver_port=45265
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:49.913648 18091 log.cc:824] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:49.916205 18091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:58:49.916575 18091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Bootstrap complete.
I20260928 17:58:49.920840 18091 raft_consensus.cc:348] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:49.921253 18091 raft_consensus.cc:729] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab64796e52449699baaa8420e70d004, State: Initialized, Role: FOLLOWER
I20260928 17:58:49.921446 18091 consensus_queue.cc:260] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:49.921535 18091 raft_consensus.cc:388] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:49.921603 18091 raft_consensus.cc:482] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:49.921658 18091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:58:49.923305 18091 raft_consensus.cc:504] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:49.923425 18091 leader_election.cc:302] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eab64796e52449699baaa8420e70d004; no voters:
I20260928 17:58:49.923604 18091 leader_election.cc:288] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 17:58:49.923731 18095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 17:58:49.923890 18091 sys_catalog.cc:564] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:49.923928 18095 raft_consensus.cc:686] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [term 2 LEADER]: Becoming Leader. State: Replica: eab64796e52449699baaa8420e70d004, State: Running, Role: LEADER
I20260928 17:58:49.924038 18095 consensus_queue.cc:237] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } }
I20260928 17:58:49.924419 18096 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "eab64796e52449699baaa8420e70d004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } } }
I20260928 17:58:49.924494 18096 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:49.924746 18097 sys_catalog.cc:455] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eab64796e52449699baaa8420e70d004. Latest consensus state: current_term: 2 leader_uuid: "eab64796e52449699baaa8420e70d004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eab64796e52449699baaa8420e70d004" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44301 } } }
I20260928 17:58:49.924825 18097 sys_catalog.cc:458] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:49.924940 18101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:49.925617 18101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:49.925793 18101 catalog_manager.cc:1259] Loaded cluster ID: 805daae4e8b9411c8abfe64c4867f677
I20260928 17:58:49.925841 18101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:49.926051 18091 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 17:58:49.929752 18101 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:49.930408 18101 catalog_manager.cc:5879] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Loaded TSK: 0
I20260928 17:58:49.930748 18101 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 17:58:50.028743 18093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:50.029053 18093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:50.029098 18093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:50.030905 18093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:50.030972 18093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:50.031015 18093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:58:50.033349 18093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:33609
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=45265
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:50.033617 18093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:50.033897 18093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:50.040887 18129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.041217 18131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.041277 18128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:50.042294 18093 server_base.cc:1034] running on GCE node
I20260928 17:58:50.042546 18093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:50.042871 18093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:50.046180 18093 hybrid_clock.cc:648] HybridClock initialized: now 1790618330046142 us; error 54 us; skew 500 ppm
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:50.049883 18093 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:58:50.050462 18093 webserver.cc:460] Webserver started at http://127.5.46.65:45265/ using document root <none> and password file <none>
I20260928 17:58:50.050726 18093 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:50.050805 18093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:50.076557 18093 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260928 17:58:50.077689 18139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.077862 18093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:50.077961 18093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "d0f1898458ef47cf8dd81f19a1299e6d"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "b838b05466239af37cdc80c23fa66766"
server_key_iv: "b6f9287684f334121ae8b253f270a27d"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:50.078485 18093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:50.111024 18093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:50.114059 18093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:50.114351 18093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:50.114682 18093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:50.115109 18093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:50.115197 18093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.115296 18093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:50.115363 18093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.126407 18093 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33609
I20260928 17:58:50.126641 18252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33609 every 8 connection(s)
I20260928 17:58:50.126998 18093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:58:50.129020 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18093
I20260928 17:58:50.129320 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:34243
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:44301
--webserver_port=42657
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:50.148228 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.132600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.65:36619)
0928 17:58:50.133965 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:50.134021 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:50.134740 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:50.135605 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:50.135610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:50.135892 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:50.136199 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.136207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.137942 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.137944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.139421 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.139425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.140657 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.140658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.140817 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.141482 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:50.141493 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:50.143483 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:50.146209 (+ 2726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.146218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.146222 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.146581 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.146885 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.146886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.146886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.147016 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.147502 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:50.147508 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:50.147666 (+ 158us) client_negotiation.cc:769] Sending connection context
0928 17:58:50.147773 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 17:58:50.147967 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1308,"thread_start_us":100,"threads_started":1}
I20260928 17:58:50.149005 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.132683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:36619 (local address 127.5.46.126:44301)
0928 17:58:50.134994 (+ 2311us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:50.135005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:50.135061 (+ 56us) server_negotiation.cc:407] Connection header received
0928 17:58:50.135222 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:50.135229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:50.135292 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:50.135436 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:50.136348 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.137787 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.139616 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.140401 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.140524 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.143641 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:50.143647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:50.143647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:50.143663 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:50.146079 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.146689 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.146694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.146694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.146811 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.147081 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.147082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.147082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.147396 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:50.147470 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:50.148662 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0928 17:58:50.148797 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2242,"thread_start_us":92,"threads_started":1}
I20260928 17:58:50.150071 18253 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:50.150177 18253 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.150372 18253 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.151288 18253 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:44301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 17:58:50.151351 18253 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.151409 18253 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:50.151786 18253 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.151880 18253 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.152206 18253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:58:50.240628 18255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:50.240871 18255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:50.240913 18255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:50.242497 18255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:50.242583 18255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:50.242638 18255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:58:50.244400 18255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:34243
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=42657
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:50.244691 18255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:50.244935 18255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:50.250453 18262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.250459 18263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.250459 18265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:50.251004 18255 server_base.cc:1034] running on GCE node
I20260928 17:58:50.251291 18255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:50.251612 18255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:50.252805 18255 hybrid_clock.cc:648] HybridClock initialized: now 1790618330252783 us; error 31 us; skew 500 ppm
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:50.256155 18255 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:58:50.256763 18255 webserver.cc:460] Webserver started at http://127.5.46.66:42657/ using document root <none> and password file <none>
I20260928 17:58:50.257046 18255 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:50.257135 18255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:50.279717 18255 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:50.280921 18273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.281235 18255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:50.281358 18255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "b8c8ab10b5224c09a1aa3a9202115034"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "8c3c7fc0fc9aa803b9fc66d1d369b464"
server_key_iv: "0c1992ba0ea4d0509fb999d34ad3f677"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:50.281944 18255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:50.299701 18255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:50.302450 18255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:50.302644 18255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:50.302932 18255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:50.303351 18255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:50.303409 18255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.303444 18255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:50.303478 18255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.311307 18255 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:34243
I20260928 17:58:50.311390 18386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:34243 every 8 connection(s)
I20260928 17:58:50.311928 18255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:50.320192 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18255
I20260928 17:58:50.320484 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.46.84:43945/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:41131
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:44301
--webserver_port=37837
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:50.326779 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.312754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:55541 (local address 127.5.46.126:44301)
0928 17:58:50.312818 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:50.312820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:50.313040 (+ 220us) server_negotiation.cc:407] Connection header received
0928 17:58:50.313594 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:50.313596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:50.313613 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:50.313695 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:50.314289 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.315090 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.316258 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.316513 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.316606 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.319301 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:50.319302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:50.319303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:50.319311 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:50.324749 (+ 5438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.325469 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.325471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.325471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.325550 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.325915 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.325916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.325916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.326084 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:50.326164 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:50.326564 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 17:58:50.326672 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 17:58:50.327030 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.312730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.66:55541)
0928 17:58:50.312901 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:50.312939 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:50.313467 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:50.313780 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:50.313784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:50.313974 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:50.314181 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.314185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.315199 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.315200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.316159 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.316161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.316647 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.316648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.316791 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.317483 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:50.317494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:50.319178 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:50.324897 (+ 5719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.324911 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.324915 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.325372 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.325646 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.325647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.325647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.325825 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.326176 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:50.326184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:50.326388 (+ 204us) client_negotiation.cc:769] Sending connection context
0928 17:58:50.326511 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 17:58:50.326706 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260928 17:58:50.327785 18387 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:50.327939 18387 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.328197 18387 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.329018 18387 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:44301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 17:58:50.329107 18387 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.329174 18387 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:50.329622 18387 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.329715 18387 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.330083 18387 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:58:50.422494 18390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:50.422802 18390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:50.422880 18390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:50.424252 18390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:50.424315 18390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:50.424364 18390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:58:50.426623 18390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.46.84:43945/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:41131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=37837
--tserver_master_addrs=127.5.46.126:44301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:50.426930 18390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:50.427240 18390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:50.435791 18396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.435840 18397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:50.436262 18399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:50.439710 18390 server_base.cc:1034] running on GCE node
I20260928 17:58:50.440062 18390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:50.440412 18390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:50.441608 18390 hybrid_clock.cc:648] HybridClock initialized: now 1790618330441592 us; error 29 us; skew 500 ppm
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:50.444381 18390 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:58:50.444936 18390 webserver.cc:460] Webserver started at http://127.5.46.67:37837/ using document root <none> and password file <none>
I20260928 17:58:50.445191 18390 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:50.445278 18390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:50.471961 18390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:50.472692 18407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.472909 18390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:50.473003 18390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "a9c89157a05842b0b9dd10cc07f89aee"
format_stamp: "Formatted at 2026-09-28 17:58:49 on dist-test-slave-z2t3"
server_key: "4bb20d17635c217f3006a24797331b4f"
server_key_iv: "a7ea29c27712d8c1d49748710ce0a57f"
server_key_version: "kuduclusterkey@0"
I20260928 17:58:50.473404 18390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:50.489365 18390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:50.491554 18390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:50.491737 18390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:50.492044 18390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:50.492429 18390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:50.492489 18390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.492533 18390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:50.492560 18390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:50.499620 18390 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:41131
I20260928 17:58:50.499675 18520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:41131 every 8 connection(s)
I20260928 17:58:50.500181 18390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:58:50 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618330, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:50.510040 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18390
I20260928 17:58:50.512413 18257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.501042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:48867 (local address 127.5.46.126:44301)
0928 17:58:50.501093 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:50.501095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:50.501383 (+ 288us) server_negotiation.cc:407] Connection header received
0928 17:58:50.502113 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:50.502114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:50.502129 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:50.502164 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:50.503058 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.503802 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.505415 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.505732 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.505784 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.508734 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:50.508736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:50.508737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:50.508746 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:50.510587 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.511172 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.511174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.511175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.511243 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.511575 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.511576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.511576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.511723 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:50.511769 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:50.512228 (+ 459us) server_negotiation.cc:299] Negotiation successful
0928 17:58:50.512320 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:58:50.512878 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.501035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:44301 (local address 127.5.46.67:48867)
0928 17:58:50.501231 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:50.501287 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:50.501972 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:50.502383 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:50.502388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:50.502651 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:50.502938 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.502945 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.503991 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.503992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.505290 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:50.505294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.505912 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.505914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:50.506072 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.506879 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:50.506890 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:50.508608 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:50.510701 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.510710 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.510714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.511050 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.511324 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:50.511324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:50.511325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:50.511487 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:50.511818 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:50.511824 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:50.512045 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 17:58:50.512512 (+ 467us) client_negotiation.cc:240] Negotiation successful
0928 17:58:50.512691 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260928 17:58:50.513419 18521 heartbeater.cc:347] Connected to a master server at 127.5.46.126:44301
I20260928 17:58:50.513530 18521 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.513733 18521 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.514515 18521 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:44301 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 17:58:50.514619 18521 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.514698 18521 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:50.515436 18521 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:50.515518 18521 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
W20260928 17:58:50.515892 18521 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 17:58:50.519165 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:50.511071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.5.46.126:44301)
0928 17:58:50.511472 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:50.511475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:50.511482 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:58:50.511503 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:50.511504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:50.511517 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:50.511581 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:50.512430 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.513231 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.514393 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:50.514583 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:50.514666 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:50.516376 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:50.516378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:50.516378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:50.516385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:50.517861 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.518308 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.518309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.518310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.518380 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:50.518626 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:50.518627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:50.518627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:50.518776 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:50.518902 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:50.519021 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 17:58:50.519088 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":71,"threads_started":1}
Sep 28 17:58:51 dist-test-slave-z2t3 krb5kdc[17059](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790618331, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:51.152382 18253 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:51.152537 18253 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:51.152891 18055 ts_manager.cc:194] Registered new tserver with Master: d0f1898458ef47cf8dd81f19a1299e6d (127.5.46.65:33609)
I20260928 17:58:51.153910 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:36619
I20260928 17:58:51.330278 18387 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:51.330463 18387 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:51.330806 18055 ts_manager.cc:194] Registered new tserver with Master: b8c8ab10b5224c09a1aa3a9202115034 (127.5.46.66:34243)
I20260928 17:58:51.331357 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:55541
Sep 28 17:58:51 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618331, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
I20260928 17:58:51.516084 18521 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:51.516249 18521 heartbeater.cc:511] Master 127.5.46.126:44301 requested a full tablet report, sending...
I20260928 17:58:51.516601 18055 ts_manager.cc:194] Registered new tserver with Master: a9c89157a05842b0b9dd10cc07f89aee (127.5.46.67:41131)
I20260928 17:58:51.517117 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:48867
I20260928 17:58:51.526525 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:58:51.536238 18542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.527665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.5.46.126:44301)
0928 17:58:51.527899 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.527902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.528023 (+ 121us) server_negotiation.cc:407] Connection header received
0928 17:58:51.528145 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.528146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.528160 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.528207 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:51.528997 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.529771 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.530803 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.531108 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.531157 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.532833 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:51.532834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:51.532835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:51.532842 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:51.534771 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:51.535209 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:51.535211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:51.535211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:51.535276 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:51.535495 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:51.535496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:51.535497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:51.535644 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:51.535727 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.536018 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.536129 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":68,"threads_started":1}
I20260928 17:58:51.539124 18055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:58:51.541904 18055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:58:51.558360 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.551242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:34243 (local address 127.0.0.1:46308)
0928 17:58:51.552728 (+ 1486us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.552737 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:51.552782 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:51.553075 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:51.553079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:51.553081 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:51.553174 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.553177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.554849 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.554850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.556892 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.556895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.558014 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.558015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.558115 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.558131 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 17:58:51.558203 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 17:58:51.558262 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"thread_start_us":73,"threads_started":1}
I20260928 17:58:51.558854 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.551723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.5.46.66:34243)
0928 17:58:51.552676 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.552682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.552833 (+ 151us) server_negotiation.cc:407] Connection header received
0928 17:58:51.552870 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.552873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.552889 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.552959 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:51.553261 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.554751 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.556994 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.557829 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.557907 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.557974 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.558717 (+ 743us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.558768 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":50,"threads_started":1}
I20260928 17:58:51.560420 18321 tablet_service.cc:1467] Processing CreateTablet for tablet 82c19511e9a343438ff4bce2fdce9136 (DEFAULT_TABLE table=test-table [id=c765c0763db84172a12e4e910c711921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:51.560837 18321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c19511e9a343438ff4bce2fdce9136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:51.565446 18555 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Bootstrap starting.
I20260928 17:58:51.566205 18555 tablet_bootstrap.cc:654] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:51.566603 18555 log.cc:824] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:51.567509 18555 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: No bootstrap required, opened a new log
I20260928 17:58:51.567574 18555 ts_tablet_manager.cc:1397] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:51.567766 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.551906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39172 (local address 127.5.46.67:41131)
0928 17:58:51.555961 (+ 4055us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.555968 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.563915 (+ 7947us) server_negotiation.cc:407] Connection header received
0928 17:58:51.563960 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.563962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.563978 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.564093 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:51.564251 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.565252 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.566582 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.567314 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.567362 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.567420 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.567642 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.567683 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4007,"thread_start_us":51,"threads_started":1}
I20260928 17:58:51.568079 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.551410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:41131 (local address 127.0.0.1:39172)
0928 17:58:51.555937 (+ 4527us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.555950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:51.556022 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:51.564094 (+ 8072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:51.564096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:51.564098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:51.564162 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.564164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.565360 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.565361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.566475 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.566478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.567484 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.567485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.567547 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.567551 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:51.567930 (+ 379us) client_negotiation.cc:240] Negotiation successful
0928 17:58:51.567983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4504,"thread_start_us":71,"threads_started":1}
I20260928 17:58:51.570034 18555 raft_consensus.cc:348] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.570219 18555 raft_consensus.cc:374] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:51.570250 18555 raft_consensus.cc:729] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c8ab10b5224c09a1aa3a9202115034, State: Initialized, Role: FOLLOWER
I20260928 17:58:51.570372 18555 consensus_queue.cc:260] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.570636 18555 ts_tablet_manager.cc:1428] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 17:58:51.570689 18387 heartbeater.cc:503] Master 127.5.46.126:44301 was elected leader, sending a full tablet report...
I20260928 17:58:51.572124 18455 tablet_service.cc:1467] Processing CreateTablet for tablet 82c19511e9a343438ff4bce2fdce9136 (DEFAULT_TABLE table=test-table [id=c765c0763db84172a12e4e910c711921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:51.572491 18455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c19511e9a343438ff4bce2fdce9136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:51.574548 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.550987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33609 (local address 127.0.0.1:35474)
0928 17:58:51.552413 (+ 1426us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.552465 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:51.552533 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:51.560286 (+ 7753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:51.560287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:51.560289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:51.560339 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.560341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.561586 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.561588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.573186 (+ 11598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.573191 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.574260 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.574262 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.574331 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.574339 (+ 8us) client_negotiation.cc:769] Sending connection context
0928 17:58:51.574372 (+ 33us) client_negotiation.cc:240] Negotiation successful
0928 17:58:51.574454 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1391,"thread_start_us":82,"threads_started":1}
I20260928 17:58:51.574995 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.551081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35474 (local address 127.5.46.65:33609)
0928 17:58:51.560009 (+ 8928us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.560014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.560028 (+ 14us) server_negotiation.cc:407] Connection header received
0928 17:58:51.560068 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.560070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.560087 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.560184 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:51.560418 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.561469 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.573314 (+ 11845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.574128 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.574749 (+ 621us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.574842 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.574872 (+ 30us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.574917 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8886,"thread_start_us":49,"threads_started":1}
I20260928 17:58:51.575721 18169 tablet_service.cc:1467] Processing CreateTablet for tablet 82c19511e9a343438ff4bce2fdce9136 (DEFAULT_TABLE table=test-table [id=c765c0763db84172a12e4e910c711921]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:51.576139 18169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82c19511e9a343438ff4bce2fdce9136. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:51.577582 18558 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Bootstrap starting.
I20260928 17:58:51.578398 18558 tablet_bootstrap.cc:654] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:51.578804 18558 log.cc:824] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:51.579659 18558 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: No bootstrap required, opened a new log
I20260928 17:58:51.579728 18558 ts_tablet_manager.cc:1397] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:51.582399 18558 raft_consensus.cc:348] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.582690 18558 raft_consensus.cc:374] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:51.582770 18558 raft_consensus.cc:729] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c89157a05842b0b9dd10cc07f89aee, State: Initialized, Role: FOLLOWER
I20260928 17:58:51.582988 18558 consensus_queue.cc:260] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.583325 18558 ts_tablet_manager.cc:1428] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260928 17:58:51.583483 18521 heartbeater.cc:503] Master 127.5.46.126:44301 was elected leader, sending a full tablet report...
I20260928 17:58:51.585165 18560 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Bootstrap starting.
I20260928 17:58:51.585927 18560 tablet_bootstrap.cc:654] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:51.586246 18560 log.cc:824] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:51.587055 18560 tablet_bootstrap.cc:492] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: No bootstrap required, opened a new log
I20260928 17:58:51.587152 18560 ts_tablet_manager.cc:1397] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 17:58:51.589267 18560 raft_consensus.cc:348] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.589450 18560 raft_consensus.cc:374] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:51.589497 18560 raft_consensus.cc:729] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f1898458ef47cf8dd81f19a1299e6d, State: Initialized, Role: FOLLOWER
I20260928 17:58:51.589643 18560 consensus_queue.cc:260] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.590016 18253 heartbeater.cc:503] Master 127.5.46.126:44301 was elected leader, sending a full tablet report...
I20260928 17:58:51.590013 18560 ts_tablet_manager.cc:1428] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260928 17:58:51.632352 18254 tablet.cc:2369] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:58:51.752017 18522 tablet.cc:2369] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 17:58:51 dist-test-slave-z2t3 krb5kdc[17059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618331, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for HTTP/127.5.46.84@KRBTEST.COM
W20260928 17:58:51.813271 18388 tablet.cc:2369] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:58:51.833431 18557 raft_consensus.cc:482] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:58:51.833611 18557 raft_consensus.cc:504] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.834040 18557 leader_election.cc:288] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f1898458ef47cf8dd81f19a1299e6d (127.5.46.65:33609), a9c89157a05842b0b9dd10cc07f89aee (127.5.46.67:41131)
I20260928 17:58:51.844849 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.834375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:42355 (local address 127.5.46.67:41131)
0928 17:58:51.834439 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.834441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.835261 (+ 820us) server_negotiation.cc:407] Connection header received
0928 17:58:51.835366 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.835368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.835380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.835408 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:51.835638 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.836417 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.839134 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.839740 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.839788 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.839799 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.844700 (+ 4901us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.844764 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 17:58:51.845188 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.834281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:41131 (local address 127.5.46.66:42355)
0928 17:58:51.835195 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.835209 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:51.835291 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:51.835473 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:51.835474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:51.835476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:51.835562 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.835565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.836896 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.836897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.839029 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.839033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.844488 (+ 5455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.844490 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.844597 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.844602 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:58:51.845013 (+ 411us) client_negotiation.cc:240] Negotiation successful
0928 17:58:51.845068 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":109,"threads_started":1}
I20260928 17:58:51.845553 18475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c19511e9a343438ff4bce2fdce9136" candidate_uuid: "b8c8ab10b5224c09a1aa3a9202115034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c89157a05842b0b9dd10cc07f89aee" is_pre_election: true
I20260928 17:58:51.845736 18475 raft_consensus.cc:2454] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c8ab10b5224c09a1aa3a9202115034 in term 0.
I20260928 17:58:51.846001 18274 leader_election.cc:302] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c89157a05842b0b9dd10cc07f89aee, b8c8ab10b5224c09a1aa3a9202115034; no voters:
I20260928 17:58:51.846154 18557 raft_consensus.cc:2781] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:58:51.846222 18557 raft_consensus.cc:482] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:58:51.846252 18557 raft_consensus.cc:3037] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:51.847185 18557 raft_consensus.cc:504] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.847345 18557 leader_election.cc:288] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [CANDIDATE]: Term 1 election: Requested vote from peers d0f1898458ef47cf8dd81f19a1299e6d (127.5.46.65:33609), a9c89157a05842b0b9dd10cc07f89aee (127.5.46.67:41131)
I20260928 17:58:51.847584 18475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c19511e9a343438ff4bce2fdce9136" candidate_uuid: "b8c8ab10b5224c09a1aa3a9202115034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c89157a05842b0b9dd10cc07f89aee"
I20260928 17:58:51.847671 18475 raft_consensus.cc:3037] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:51.848565 18475 raft_consensus.cc:2454] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c8ab10b5224c09a1aa3a9202115034 in term 1.
I20260928 17:58:51.848778 18274 leader_election.cc:302] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c89157a05842b0b9dd10cc07f89aee, b8c8ab10b5224c09a1aa3a9202115034; no voters:
I20260928 17:58:51.848873 18557 raft_consensus.cc:2781] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:51.849042 18557 raft_consensus.cc:686] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 1 LEADER]: Becoming Leader. State: Replica: b8c8ab10b5224c09a1aa3a9202115034, State: Running, Role: LEADER
I20260928 17:58:51.849160 18557 consensus_queue.cc:237] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } }
I20260928 17:58:51.850134 18055 catalog_manager.cc:5495] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c8ab10b5224c09a1aa3a9202115034 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "b8c8ab10b5224c09a1aa3a9202115034" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8c8ab10b5224c09a1aa3a9202115034" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 } health_report { overall_health: UNKNOWN } } }
I20260928 17:58:51.854235 18550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.834693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:55259 (local address 127.5.46.65:33609)
0928 17:58:51.834739 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.834741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.843823 (+ 9082us) server_negotiation.cc:407] Connection header received
0928 17:58:51.843991 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.843993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.844009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.844044 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:51.844305 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.845042 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.853186 (+ 8144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.853808 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.853892 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.853904 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.854113 (+ 209us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.854161 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:58:51.854720 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.834599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33609 (local address 127.5.46.66:55259)
0928 17:58:51.843707 (+ 9108us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.843725 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:51.843893 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:51.844125 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:51.844127 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:51.844129 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:51.844219 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.844221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.851812 (+ 7591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.851813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.853055 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:51.853060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.853943 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.853944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:51.854027 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.854031 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:51.854526 (+ 495us) client_negotiation.cc:240] Negotiation successful
0928 17:58:51.854633 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9076,"thread_start_us":82,"threads_started":1}
I20260928 17:58:51.855221 18203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c19511e9a343438ff4bce2fdce9136" candidate_uuid: "b8c8ab10b5224c09a1aa3a9202115034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" is_pre_election: true
I20260928 17:58:51.855229 18204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82c19511e9a343438ff4bce2fdce9136" candidate_uuid: "b8c8ab10b5224c09a1aa3a9202115034" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f1898458ef47cf8dd81f19a1299e6d"
I20260928 17:58:51.855407 18203 raft_consensus.cc:2454] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c8ab10b5224c09a1aa3a9202115034 in term 0.
I20260928 17:58:51.916430 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.912891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.5.46.66:34243)
0928 17:58:51.912933 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:51.912935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:51.912943 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:58:51.912983 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:51.912984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:51.913007 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:51.913080 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:58:51.913301 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.914062 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.915232 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:51.915532 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:51.915596 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:51.915828 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:58:51.916007 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:58:51.916086 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:51.916320 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 17:58:51.916343 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:58:51.918529 18475 raft_consensus.cc:1261] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 1 FOLLOWER]: Refusing update from remote peer b8c8ab10b5224c09a1aa3a9202115034: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:51.918865 18568 consensus_queue.cc:1035] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c89157a05842b0b9dd10cc07f89aee" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 41131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:51.921130 18204 raft_consensus.cc:3037] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:51.922214 18204 raft_consensus.cc:1261] T 82c19511e9a343438ff4bce2fdce9136 P d0f1898458ef47cf8dd81f19a1299e6d [term 1 FOLLOWER]: Refusing update from remote peer b8c8ab10b5224c09a1aa3a9202115034: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:51.922569 18557 consensus_queue.cc:1035] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f1898458ef47cf8dd81f19a1299e6d" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:51.923295 18571 mvcc.cc:205] Tried to move back new op lower bound from 7334372687535169536 to 7334372687254953984. Current Snapshot: MvccSnapshot[applied={T|T < 7334372687535169536}]
I20260928 17:58:51.923900 18572 mvcc.cc:205] Tried to move back new op lower bound from 7334372687535169536 to 7334372687254953984. Current Snapshot: MvccSnapshot[applied={T|T < 7334372687535169536}]
I20260928 17:58:51.925026 18575 mvcc.cc:205] Tried to move back new op lower bound from 7334372687535169536 to 7334372687254953984. Current Snapshot: MvccSnapshot[applied={T|T < 7334372687535169536}]
I20260928 17:58:51.926311 18055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:51.926414 18055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:51.927266 18055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eab64796e52449699baaa8420e70d004: Sending DeleteTablet for 3 replicas of tablet 82c19511e9a343438ff4bce2fdce9136
I20260928 17:58:51.928126 18169 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c19511e9a343438ff4bce2fdce9136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:51 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:35474
I20260928 17:58:51.928344 18321 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c19511e9a343438ff4bce2fdce9136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:51 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46308
I20260928 17:58:51.928565 18455 tablet_service.cc:1514] Processing DeleteTablet for tablet 82c19511e9a343438ff4bce2fdce9136 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:51 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:39172
I20260928 17:58:51.929101 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18093
I20260928 17:58:51.931964 18582 tablet_replica.cc:330] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: stopping tablet replica
I20260928 17:58:51.932122 18582 raft_consensus.cc:2229] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:58:51.932327 18582 raft_consensus.cc:2258] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:51.928815 18583 tablet_replica.cc:330] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: stopping tablet replica
I20260928 17:58:51.937223 18583 raft_consensus.cc:2229] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:51.937353 18582 ts_tablet_manager.cc:1905] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:51.937397 18583 raft_consensus.cc:2258] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:51.937932 18583 ts_tablet_manager.cc:1905] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:51.939648 18583 ts_tablet_manager.cc:1918] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:51.939735 18583 log.cc:1196] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/82c19511e9a343438ff4bce2fdce9136
I20260928 17:58:51.939918 18582 ts_tablet_manager.cc:1918] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:51.940048 18583 ts_tablet_manager.cc:1939] T 82c19511e9a343438ff4bce2fdce9136 P a9c89157a05842b0b9dd10cc07f89aee: Deleting consensus metadata
I20260928 17:58:51.940054 18582 log.cc:1196] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/82c19511e9a343438ff4bce2fdce9136
W20260928 17:58:51.940371 18042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:33609 (error 108)
I20260928 17:58:51.940917 18039 catalog_manager.cc:4841] TS a9c89157a05842b0b9dd10cc07f89aee (127.5.46.67:41131): tablet 82c19511e9a343438ff4bce2fdce9136 (table test-table [id=c765c0763db84172a12e4e910c711921]) successfully deleted
I20260928 17:58:51.940950 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18255
I20260928 17:58:51.941162 18582 ts_tablet_manager.cc:1939] T 82c19511e9a343438ff4bce2fdce9136 P b8c8ab10b5224c09a1aa3a9202115034: Deleting consensus metadata
I20260928 17:58:51.942008 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.941868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33609 (local address 127.0.0.1:35484)
0928 17:58:51.941931 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.941964 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33609: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 17:58:51.942178 18042 catalog_manager.cc:4584] TS d0f1898458ef47cf8dd81f19a1299e6d (127.5.46.65:33609): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82c19511e9a343438ff4bce2fdce9136: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33609: connect: Connection refused (error 111)
I20260928 17:58:51.945924 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:51.945808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:34243 (local address 127.0.0.1:46318)
0928 17:58:51.945854 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:51.945878 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.66:34243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260928 17:58:51.946292 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18390
I20260928 17:58:51.951656 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18022
I20260928 17:58:51.957396 5305 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 17:58:52.302386 5305 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 17:58:52.302493 5305 mini_ranger.cc:66] Stopping Ranger...
I20260928 17:58:52.647635 5305 mini_ranger.cc:68] Stopped Ranger
2026-09-28 17:58:52.647 UTC [17089] LOG: received smart shutdown request
2026-09-28 17:58:52.652 UTC [17089] LOG: background worker "logical replication launcher" (PID 17098) exited with exit code 1
2026-09-28 17:58:52.652 UTC [17093] LOG: shutting down
2026-09-28 17:58:52.852 UTC [17089] LOG: database system is shut down
2026-09-28T17:58:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49373 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:58:53 dist-test-slave-z2t3 krb5kdc[18593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:58:53 dist-test-slave-z2t3 krb5kdc[18593](info): set up 2 sockets
Sep 28 17:58:53 dist-test-slave-z2t3 krb5kdc[18593](info): commencing operation
krb5kdc: starting...
W20260928 17:58:54.019907 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:58:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:58:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:58:54.146981 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42127
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:54.231014 18606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:54.231248 18606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:54.231273 18606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:54.232539 18606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:54.232642 18606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:54.232658 18606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:54.232671 18606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:54.232684 18606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:54.234710 18606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:54.235030 18606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:54.235334 18606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:54.240654 18614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:54.240670 18612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:54.240688 18606 server_base.cc:1034] running on GCE node
W20260928 17:58:54.240681 18611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:54.241194 18606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:54.241523 18606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:54.242700 18606 hybrid_clock.cc:648] HybridClock initialized: now 1790618334242676 us; error 37 us; skew 500 ppm
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:54.250958 18606 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:54.251421 18606 webserver.cc:460] Webserver started at http://127.5.46.126:36751/ using document root <none> and password file <none>
I20260928 17:58:54.251631 18606 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:54.251677 18606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:54.251773 18606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:54.252972 18606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "8670d18243b84b6b859b069e08c488af"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "996d1bf90403f6e05d805f8a8944e7ed"
server_key_iv: "69ef80df431afb5f6f874acf2af38e76"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.253291 18606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "8670d18243b84b6b859b069e08c488af"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "996d1bf90403f6e05d805f8a8944e7ed"
server_key_iv: "69ef80df431afb5f6f874acf2af38e76"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.254606 18606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 17:58:54.255404 18621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:54.255671 18606 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:58:54.255757 18606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "8670d18243b84b6b859b069e08c488af"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "996d1bf90403f6e05d805f8a8944e7ed"
server_key_iv: "69ef80df431afb5f6f874acf2af38e76"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.255810 18606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:54.264081 18606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:54.266566 18606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:54.266697 18606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:54.270992 18606 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:42127
I20260928 17:58:54.271028 18673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:42127 every 8 connection(s)
I20260928 17:58:54.271431 18606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:54.272189 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18606
I20260928 17:58:54.272306 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:58:54.272531 5305 external_mini_cluster.cc:1428] Setting key b34731d32e29dcca77aa75a0a36ecdc7
I20260928 17:58:54.272617 18674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:54.274781 18674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.46.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.125@KRBTEST.COM" created.
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:54.288933 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.275497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42127 (local address 127.0.0.1:46766)
0928 17:58:54.275682 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.275729 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.276292 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.276648 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.276651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.276952 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.277113 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.277117 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.278801 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.278802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.279504 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.279506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.280288 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.280289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.280356 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.280935 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.280944 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.284686 (+ 3742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.287210 (+ 2524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.287218 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.287220 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.287438 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.287735 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.287736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.287736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.287831 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.288426 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.288432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.288506 (+ 74us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.288577 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.288739 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260928 17:58:54.289018 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.275525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46766 (local address 127.5.46.126:42127)
0928 17:58:54.275768 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.275773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.275806 (+ 33us) server_negotiation.cc:407] Connection header received
0928 17:58:54.276468 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.276471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.276495 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.276604 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.277251 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.278687 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.279601 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.280170 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.280307 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.284807 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.284815 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.284816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.284823 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.287096 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.287532 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.287536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.287537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.287661 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.287975 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.287977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.287977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.288343 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.288437 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.288659 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.288837 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":214,"thread_start_us":145,"threads_started":1}
I20260928 17:58:54.289821 18674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.290227 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.290098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47846)
0928 17:58:54.290168 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.290208 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260928 17:58:54.290359 18623 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
I20260928 17:58:54.290861 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.290757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47860)
0928 17:58:54.290803 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.290829 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 17:58:54.291007 18674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:42969: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
I20260928 17:58:54.303685 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:42969
--webserver_interface=127.5.46.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:54.340216 18674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } attempt: 1
I20260928 17:58:54.340709 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.340573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47874)
0928 17:58:54.340637 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.340680 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 17:58:54.341179 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.341055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47876)
0928 17:58:54.341121 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.341157 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 17:58:54.341301 18674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:42969: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
W20260928 17:58:54.387144 18681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:54.387409 18681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:54.387450 18681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:54.388670 18681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:54.388744 18681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:54.388769 18681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:54.388792 18681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:54.388820 18681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:54.390327 18681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.46.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:54.390614 18681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:54.390852 18681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:54.395943 18689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:54.395918 18686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:54.396118 18681 server_base.cc:1034] running on GCE node
W20260928 17:58:54.395968 18687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:54.396420 18681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:54.396749 18681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:54.397926 18681 hybrid_clock.cc:648] HybridClock initialized: now 1790618334397922 us; error 38 us; skew 500 ppm
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:54.406296 18681 init.cc:377] Logged in from keytab as kudu/127.5.46.125@KRBTEST.COM (short username kudu)
I20260928 17:58:54.406832 18681 webserver.cc:460] Webserver started at http://127.5.46.125:37827/ using document root <none> and password file <none>
I20260928 17:58:54.407073 18681 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:54.407122 18681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:54.407267 18681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:54.408471 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data/instance:
uuid: "fedbd79e09684e09b1751ab00cc48290"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "e9958433c8ccd9c842a2fd754d3ceb57"
server_key_iv: "17c7db400fb39eb5dc7f6bed936d2937"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.408850 18681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal/instance:
uuid: "fedbd79e09684e09b1751ab00cc48290"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "e9958433c8ccd9c842a2fd754d3ceb57"
server_key_iv: "17c7db400fb39eb5dc7f6bed936d2937"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.409457 18674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } attempt: 2
I20260928 17:58:54.410037 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.409874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47880)
0928 17:58:54.409948 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.409993 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 17:58:54.410506 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.410385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47892)
0928 17:58:54.410450 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.410480 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 17:58:54.410497 18681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
W20260928 17:58:54.410676 18674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:42969: Network error: Client connection negotiation failed: client connection to 127.5.46.125:42969: connect: Connection refused (error 111)
I20260928 17:58:54.411375 18696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:54.411614 18681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:54.411724 18681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "fedbd79e09684e09b1751ab00cc48290"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "e9958433c8ccd9c842a2fd754d3ceb57"
server_key_iv: "17c7db400fb39eb5dc7f6bed936d2937"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.411823 18681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:54.426023 18681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:54.428115 18681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:54.428297 18681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:54.432467 18681 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.125:42969
I20260928 17:58:54.432544 18748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.125:42969 every 8 connection(s)
I20260928 17:58:54.432924 18681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
I20260928 17:58:54.433874 18749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:54.435482 18749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.439287 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18681
I20260928 17:58:54.439402 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-1/wal/instance
I20260928 17:58:54.439596 5305 external_mini_cluster.cc:1428] Setting key c3bfae19e2e6f3e26888d75f6716c17d
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.124@KRBTEST.COM; defaulting to no policy
I20260928 17:58:54.449004 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.436284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.5.46.126:42127)
0928 17:58:54.436351 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.436353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.436552 (+ 199us) server_negotiation.cc:407] Connection header received
0928 17:58:54.437034 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.437035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.437048 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.437161 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.438230 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.439045 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.440645 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.441365 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.441470 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.444929 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.444952 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.444953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.444962 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.447002 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.447671 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.447673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.447674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.447749 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.448163 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.448164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.448165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.448354 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.448423 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.448831 (+ 408us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.448933 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 17:58:54.449266 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.436256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42127 (local address 127.0.0.1:46776)
0928 17:58:54.436423 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.436466 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.436921 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.437255 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.437259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.437586 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.438116 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.438122 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.439171 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.439172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.440523 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.440526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.441510 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.441511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.441759 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.442540 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.442552 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.444785 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.447106 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.447122 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.447126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.447521 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.447924 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.447925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.447925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.448091 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.448438 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.448446 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.448650 (+ 204us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.448773 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.448970 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260928 17:58:54.449906 18749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.124@KRBTEST.COM" created.
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 17:58:54.459782 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.450190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47894)
0928 17:58:54.450248 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.450261 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.450310 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.450577 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.450578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.450737 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.450794 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.450796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.451949 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.451950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.452777 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.452779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.453647 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.453648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.453735 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.454254 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.454260 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.455993 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.458495 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.458496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.458497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.458707 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.458977 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.458978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.458978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.459037 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.459448 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.459449 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.459491 (+ 42us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.459555 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.459650 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 17:58:54.460353 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.450233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.5.46.125:42969)
0928 17:58:54.450442 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.450446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.450463 (+ 17us) server_negotiation.cc:407] Connection header received
0928 17:58:54.450479 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.450481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.450492 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.450577 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.450879 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.451818 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.452893 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.453562 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.453646 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.456085 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.456095 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.456096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.456103 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.458406 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.458792 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.458796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.458796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.458853 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.459100 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.459101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.459101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.459345 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.459425 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.460120 (+ 695us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.460241 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":111,"threads_started":1}
I20260928 17:58:54.461102 18749 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.461699 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.461507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42630)
0928 17:58:54.461570 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.461666 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 17:58:54.461952 18697 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
I20260928 17:58:54.462558 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.462431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42646)
0928 17:58:54.462465 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.462526 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260928 17:58:54.462702 18749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:39997: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
I20260928 17:58:54.472046 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:39997
--webserver_interface=127.5.46.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:33019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:54.501856 18674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } attempt: 3
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 17:58:54.509665 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.502317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.5.46.125:42969)
0928 17:58:54.502390 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.502392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.502404 (+ 12us) server_negotiation.cc:407] Connection header received
0928 17:58:54.502457 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.502457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.502476 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.502543 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.502983 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.503521 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.504310 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.504529 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.504638 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.506880 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.506882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.506882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.506898 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.508469 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.508859 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.508861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.508861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.508920 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.509127 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.509128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.509129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.509250 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.509322 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.509484 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.509567 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 17:58:54.509670 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.502226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47900)
0928 17:58:54.502302 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.502317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.502390 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.502548 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.502549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.502783 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.502882 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.502884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.503617 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.503619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.504221 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.504224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.504646 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.504647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.504728 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.505251 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.505256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.506774 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.508578 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.508580 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.508580 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.508770 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.508994 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.508995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.508995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.509059 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.509326 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.509327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.509375 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.509459 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.509564 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 17:58:54.510051 18674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.510412 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.510308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42658)
0928 17:58:54.510360 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.510387 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 17:58:54.510924 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.510711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42668)
0928 17:58:54.510860 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.510898 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260928 17:58:54.511073 18674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:39997: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
I20260928 17:58:54.511953 18749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } attempt: 1
I20260928 17:58:54.512378 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.512251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42680)
0928 17:58:54.512306 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.512345 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 17:58:54.512795 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.512703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42684)
0928 17:58:54.512747 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.512775 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 17:58:54.512907 18749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:39997: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
I20260928 17:58:54.542238 18674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } attempt: 1
I20260928 17:58:54.542815 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.542663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42698)
0928 17:58:54.542738 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.542783 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 17:58:54.543278 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.543157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42714)
0928 17:58:54.543225 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.543256 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 17:58:54.543413 18674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:39997: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
W20260928 17:58:54.561657 18756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:54.561885 18756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:54.561909 18756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:54.563122 18756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:54.563170 18756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:54.563185 18756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:54.563198 18756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:54.563210 18756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:54.564852 18756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:33019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42127,127.5.46.125:42969,127.5.46.124:39997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:39997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.46.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:54.565125 18756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:54.565367 18756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:54.570323 18764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:54.570307 18761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:54.570290 18762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:54.570351 18756 server_base.cc:1034] running on GCE node
I20260928 17:58:54.570776 18756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:54.571136 18756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:54.572328 18756 hybrid_clock.cc:648] HybridClock initialized: now 1790618334572315 us; error 35 us; skew 500 ppm
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:54.575171 18756 init.cc:377] Logged in from keytab as kudu/127.5.46.124@KRBTEST.COM (short username kudu)
I20260928 17:58:54.575670 18756 webserver.cc:460] Webserver started at http://127.5.46.124:41115/ using document root <none> and password file <none>
I20260928 17:58:54.575973 18756 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:54.576053 18756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:54.576220 18756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:54.577419 18756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data/instance:
uuid: "9e37561821534243be6ae03e26da1766"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "bb5530fd31e2928773215b8d898fc5ba"
server_key_iv: "804e31b6fc090b8d53a2172166e1a047"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.577800 18756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal/instance:
uuid: "9e37561821534243be6ae03e26da1766"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "bb5530fd31e2928773215b8d898fc5ba"
server_key_iv: "804e31b6fc090b8d53a2172166e1a047"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.579217 18756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:58:54.580094 18771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:54.580291 18756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:54.580410 18756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "9e37561821534243be6ae03e26da1766"
format_stamp: "Formatted at 2026-09-28 17:58:54 on dist-test-slave-z2t3"
server_key: "bb5530fd31e2928773215b8d898fc5ba"
server_key_iv: "804e31b6fc090b8d53a2172166e1a047"
server_key_version: "encryptionkey@0"
I20260928 17:58:54.580490 18756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:54.581082 18749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } attempt: 2
I20260928 17:58:54.581470 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.581353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42730)
0928 17:58:54.581407 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.581444 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 17:58:54.581784 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.581676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42742)
0928 17:58:54.581747 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.581768 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260928 17:58:54.581871 18749 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:39997: Network error: Client connection negotiation failed: client connection to 127.5.46.124:39997: connect: Connection refused (error 111)
I20260928 17:58:54.607825 18756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:54.610518 18756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:54.610709 18756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:54.614914 18756 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.124:39997
I20260928 17:58:54.614957 18823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.124:39997 every 8 connection(s)
I20260928 17:58:54.615372 18756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
I20260928 17:58:54.616357 18824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:54.617971 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18756
I20260928 17:58:54.618093 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790618177091767-5305-0/minicluster-data/master-2/wal/instance
I20260928 17:58:54.617928 18824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.618300 5305 external_mini_cluster.cc:1428] Setting key 917f1ad71bc8b8ad590b71a7a3a5ef90
I20260928 17:58:54.618563 18674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } attempt: 2
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 17:58:54.629035 18677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.618864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42760)
0928 17:58:54.618924 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.618936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.618982 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.619719 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.619721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.619948 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.620006 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.620008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.621214 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.621215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.622026 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.622028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.622635 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.622697 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.623222 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.623227 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.625413 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.627779 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.627780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.627780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.628061 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.628276 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.628277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.628342 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.628690 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.628690 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.628734 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.628795 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.628897 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 17:58:54.629213 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.619050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.5.46.126:42127)
0928 17:58:54.619085 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.619087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.619093 (+ 6us) server_negotiation.cc:407] Connection header received
0928 17:58:54.619682 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.619683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.619696 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.619747 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.620381 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.621146 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622030 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.622334 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622403 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.625135 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.625137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.625137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.625146 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.627049 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.627658 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.627660 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.627661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.627724 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628114 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.628116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.628116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.628329 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.628376 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.628738 (+ 362us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.628827 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 17:58:54.629441 18827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.618721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.5.46.124:39997)
0928 17:58:54.618906 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.618917 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.619100 (+ 183us) server_negotiation.cc:407] Connection header received
0928 17:58:54.619569 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.619576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.619601 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.619749 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.620499 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.621252 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622291 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.622593 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622920 (+ 327us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.625637 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.625638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.625638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.625645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.627958 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628343 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.628344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.628344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.628392 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628596 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.628597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.628597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.628735 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.628873 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.628960 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.629201 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
I20260928 17:58:54.629520 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.619152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.5.46.124:39997)
0928 17:58:54.619291 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.619292 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.619301 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:58:54.619575 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.619576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.619602 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.620086 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.620131 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.621118 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622097 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.622539 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622632 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.625501 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.625509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.625509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.625515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.627676 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628135 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.628139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.628139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.628201 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.628404 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.628406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.628406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.628615 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.628700 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.629324 (+ 624us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.629399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
I20260928 17:58:54.629726 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.618710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42127 (local address 127.0.0.1:46782)
0928 17:58:54.618897 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.618939 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.619571 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.619814 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.619817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.620125 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.620270 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.620275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.621249 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.621250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.621938 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.621940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.622413 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.622414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.622542 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.623068 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.623076 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.625042 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.627170 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.627175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.627177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.627561 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.627806 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.627807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.627808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.627984 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.628424 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.628432 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.628595 (+ 163us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.628966 (+ 371us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.629633 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":537,"thread_start_us":96,"threads_started":1}
I20260928 17:58:54.630288 18824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.631716 18674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: Bootstrap starting.
I20260928 17:58:54.632484 18674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:54.632767 18674 log.cc:824] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:54.633628 18674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: No bootstrap required, opened a new log
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:54.635439 18674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.635553 18674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:54.635576 18674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8670d18243b84b6b859b069e08c488af, State: Initialized, Role: FOLLOWER
I20260928 17:58:54.635759 18674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.636086 18831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.636220 18831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:58:54.636159 18674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 17:58:54.639215 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.630408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46798 (local address 127.5.46.126:42127)
0928 17:58:54.630441 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.630443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.630450 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:58:54.630471 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.630471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.630481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.630535 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.631145 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.631830 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.632849 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.633122 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.633185 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.635693 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.635695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.635737 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.635743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.637499 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.637906 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.637908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.637908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.637967 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.638191 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.638192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.638192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.638338 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.638520 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.638747 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.638828 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 17:58:54.639554 18842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:58:54.639638 18842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:58:54.641134 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.630528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:42969 (local address 127.0.0.1:47908)
0928 17:58:54.630628 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.630638 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.630679 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.630806 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.630806 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.630947 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.630995 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.630996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.631971 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.631972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.632464 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.632466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.632847 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.632848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.632920 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.635958 (+ 3038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.635963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.637813 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.639936 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.639937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.639938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.640172 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.640397 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.640398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.640398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.640472 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.640757 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.640758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.640816 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.640892 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.641017 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260928 17:58:54.641170 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.630592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.5.46.125:42969)
0928 17:58:54.630633 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.630635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.630666 (+ 31us) server_negotiation.cc:407] Connection header received
0928 17:58:54.630737 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.630738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.630748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.630802 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.631077 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.631822 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.632557 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.632771 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.632853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.637958 (+ 5105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.637960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.637960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.637969 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.639797 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.640254 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.640255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.640255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.640318 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.640544 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.640545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.640546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.640690 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.640762 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.640954 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.641052 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 17:58:54.641469 18824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } has no permanent_uuid. Determining permanent_uuid...
I20260928 17:58:54.639430 18606 master_runner.cc:186] Error getting master registration for 127.5.46.125:42969: OK, instance_id { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" instance_seqno: 1790618334431207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 17:58:54.642589 18606 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 17:58:54.649307 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.641850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.5.46.124:39997)
0928 17:58:54.641895 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.641897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.641905 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:58:54.641923 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.641924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.641934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.641997 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.642268 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.642820 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.643721 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.644033 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.644065 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.646176 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.646178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.646178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.646186 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.648062 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.648442 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.648443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.648443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.648489 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.648665 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.648666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.648667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.648857 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.648901 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.649096 (+ 195us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.649189 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 17:58:54.649585 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.641744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42776)
0928 17:58:54.641813 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.641822 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.641853 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.642013 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.642014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.642173 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.642211 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.642212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.642907 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.642908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.643637 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.643640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.644125 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.644125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.644154 (+ 29us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.644532 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.644537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.646078 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.648158 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.648159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.648159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.648368 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.648552 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.648552 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.648553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.648600 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.648945 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.648946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.648981 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.649072 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.649255 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":80}
I20260928 17:58:54.650477 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.639789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.5.46.125:42969)
0928 17:58:54.639948 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.639950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.639959 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:58:54.639979 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.639980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.639999 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.640076 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.640718 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.642111 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.643014 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.643348 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.643418 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.646714 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.646716 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.646716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.646723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.648870 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.649301 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.649303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.649303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.649363 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.649571 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.649572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.649572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.649700 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.649860 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.650243 (+ 383us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.650345 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260928 17:58:54.651238 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: Bootstrap starting.
I20260928 17:58:54.651975 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:58:54.652118 18824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:54.652400 18824 log.cc:824] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:54.653223 18824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: No bootstrap required, opened a new log
I20260928 17:58:54.654800 18824 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.654953 18824 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:54.654997 18824 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e37561821534243be6ae03e26da1766, State: Initialized, Role: FOLLOWER
I20260928 17:58:54.655136 18824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.655385 18851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.655444 18824 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:54.655517 18851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 17:58:54.658320 18862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:58:54.658380 18862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:58:54.659893 18756 master_runner.cc:186] Error getting master registration for 127.5.46.125:42969: OK, instance_id { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" instance_seqno: 1790618334431207 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 17:58:54.662635 18756 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 17:58:54.663775 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.653679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.5.46.124:39997)
0928 17:58:54.655499 (+ 1820us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.655502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.655512 (+ 10us) server_negotiation.cc:407] Connection header received
0928 17:58:54.655532 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.655533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.655543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.655618 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.656425 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.657130 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.657893 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.658844 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.658903 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.660686 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.660688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.660688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.660695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.662396 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.662889 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.662890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.662891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.662951 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.663167 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.663168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.663168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.663288 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.663407 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.663503 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.663583 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1799}
I20260928 17:58:54.670487 18678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.664099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.5.46.126:42127)
0928 17:58:54.664234 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.664236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.664332 (+ 96us) server_negotiation.cc:407] Connection header received
0928 17:58:54.664358 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.664359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.664371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.664418 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.664963 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.665438 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.666114 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.666324 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.666412 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.667685 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.667686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.667687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.667694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.669280 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.669660 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.669662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.669662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.669723 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.669944 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.669945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.669945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.670070 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.670185 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.670298 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.670380 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260928 17:58:54.673014 18749 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } attempt: 3
I20260928 17:58:54.677944 18844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.664384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.5.46.125:42969)
0928 17:58:54.664425 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.664427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.670525 (+ 6098us) server_negotiation.cc:407] Connection header received
0928 17:58:54.670942 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.670944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.670962 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.670994 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.671513 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.672109 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.672794 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.673063 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.673135 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.674797 (+ 1662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.674798 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.674798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.674806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.676592 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.677000 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.677001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.677001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.677061 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.677303 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.677304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.677305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.677441 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.677518 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.677764 (+ 246us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.677839 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 17:58:54 dist-test-slave-z2t3 krb5kdc[18593](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618334, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 17:58:54.682085 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.673289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:39997 (local address 127.0.0.1:42798)
0928 17:58:54.673340 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:54.673354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:54.673403 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:54.673896 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:54.673897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:54.674079 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:54.674124 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.674126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.675447 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.675448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.676618 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:54.676620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.677107 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.677108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:54.677168 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.677600 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:54.677605 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:54.679149 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:54.680972 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.680973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.680974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.681191 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.681435 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:54.681436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:54.681436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:54.681503 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:54.681813 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:54.681814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:54.681858 (+ 44us) client_negotiation.cc:769] Sending connection context
0928 17:58:54.681905 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 17:58:54.682002 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260928 17:58:54.682513 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:54.673347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.5.46.124:39997)
0928 17:58:54.673390 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:54.673392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:54.673400 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:58:54.673465 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:54.673466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:54.673477 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:54.674212 (+ 735us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:54.674271 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.675349 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.676719 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:54.677021 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:54.679290 (+ 2269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:54.679335 (+ 45us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:54.679336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:54.679336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:54.679340 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:54.680824 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.681297 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.681298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.681298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.681359 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:54.681577 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:54.681578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:54.681578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:54.681730 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:54.682308 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:54.682350 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 17:58:54.682424 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:58:54.682880 18851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:58:54.682979 18851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.683226 18851 leader_election.cc:288] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8670d18243b84b6b859b069e08c488af (127.5.46.126:42127), fedbd79e09684e09b1751ab00cc48290 (127.5.46.125:42969)
I20260928 17:58:54.684211 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: Bootstrap starting.
I20260928 17:58:54.684262 18648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e37561821534243be6ae03e26da1766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8670d18243b84b6b859b069e08c488af" is_pre_election: true
I20260928 17:58:54.684399 18648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e37561821534243be6ae03e26da1766 in term 0.
I20260928 17:58:54.684829 18773 leader_election.cc:302] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8670d18243b84b6b859b069e08c488af, 9e37561821534243be6ae03e26da1766; no voters:
I20260928 17:58:54.684975 18851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:58:54.685041 18851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:58:54.685036 18749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:54.685070 18851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:54.685349 18749 log.cc:824] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:54.686147 18851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.686277 18851 leader_election.cc:288] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [CANDIDATE]: Term 1 election: Requested vote from peers 8670d18243b84b6b859b069e08c488af (127.5.46.126:42127), fedbd79e09684e09b1751ab00cc48290 (127.5.46.125:42969)
I20260928 17:58:54.686549 18648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e37561821534243be6ae03e26da1766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8670d18243b84b6b859b069e08c488af"
I20260928 17:58:54.686661 18648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:54.687518 18648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e37561821534243be6ae03e26da1766 in term 1.
I20260928 17:58:54.687706 18773 leader_election.cc:302] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8670d18243b84b6b859b069e08c488af, 9e37561821534243be6ae03e26da1766; no voters:
I20260928 17:58:54.687879 18851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:54.688042 18851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [term 1 LEADER]: Becoming Leader. State: Replica: 9e37561821534243be6ae03e26da1766, State: Running, Role: LEADER
I20260928 17:58:54.688149 18851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.688558 18866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e37561821534243be6ae03e26da1766. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.688853 18866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:54.690203 18868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:54.690388 18868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:54.690495 18749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: No bootstrap required, opened a new log
I20260928 17:58:54.692128 18648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [term 1 FOLLOWER]: Refusing update from remote peer 9e37561821534243be6ae03e26da1766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:54.692493 18851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:54.692687 18749 raft_consensus.cc:348] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.692821 18749 raft_consensus.cc:374] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:54.692859 18749 raft_consensus.cc:729] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fedbd79e09684e09b1751ab00cc48290, State: Initialized, Role: FOLLOWER
I20260928 17:58:54.692982 18749 consensus_queue.cc:260] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } }
I20260928 17:58:54.693322 18749 sys_catalog.cc:564] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:54.693440 18721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:54.694963 18723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e37561821534243be6ae03e26da1766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fedbd79e09684e09b1751ab00cc48290" is_pre_election: true
I20260928 17:58:54.695242 18722 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9e37561821534243be6ae03e26da1766" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fedbd79e09684e09b1751ab00cc48290"
I20260928 17:58:54.694967 18721 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [term 1 FOLLOWER]: Refusing update from remote peer 9e37561821534243be6ae03e26da1766: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:54.696310 18831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e37561821534243be6ae03e26da1766. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.696398 18831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:58:54.696712 18851 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:54.699095 18866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.699185 18866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:54.699671 18868 catalog_manager.cc:1347] Generated new cluster ID: 60e8bdd3f2de452fae9987cf866f7801
I20260928 17:58:54.699747 18868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:54.700191 18870 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.700521 18870 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:58:54.700459 18884 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e37561821534243be6ae03e26da1766. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.700657 18884 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 17:58:54.701169 18883 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:58:54.701252 18883 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:58:54.704468 18884 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.704545 18884 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:58:54.705118 18885 mvcc.cc:205] Tried to move back new op lower bound from 7334372698895720448 to 7334372698883334144. Current Snapshot: MvccSnapshot[applied={T|T < 7334372698895720448}]
I20260928 17:58:54.705668 18851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.705780 18851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:54.706462 18831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e37561821534243be6ae03e26da1766" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8670d18243b84b6b859b069e08c488af" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42127 } } peers { permanent_uuid: "fedbd79e09684e09b1751ab00cc48290" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 42969 } } peers { permanent_uuid: "9e37561821534243be6ae03e26da1766" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 39997 } } }
I20260928 17:58:54.706548 18831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af [sys.catalog]: This master's current role is: FOLLOWER
I20260928 17:58:54.734166 18868 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:58:54.734938 18868 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:54.748217 18868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e37561821534243be6ae03e26da1766: Generated new TSK 0
I20260928 17:58:54.748462 18868 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:58:55.640158 18842 catalog_manager.cc:1259] Loaded cluster ID: 60e8bdd3f2de452fae9987cf866f7801
I20260928 17:58:55.640228 18842 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: loading cluster ID for follower catalog manager: success
I20260928 17:58:55.641057 18842 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: acquiring CA information for follower catalog manager: success
I20260928 17:58:55.641325 18842 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8670d18243b84b6b859b069e08c488af: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 17:58:55.701774 18883 catalog_manager.cc:1259] Loaded cluster ID: 60e8bdd3f2de452fae9987cf866f7801
I20260928 17:58:55.701857 18883 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: loading cluster ID for follower catalog manager: success
I20260928 17:58:55.702755 18883 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: acquiring CA information for follower catalog manager: success
I20260928 17:58:55.703084 18883 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fedbd79e09684e09b1751ab00cc48290: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 17:58:55.805608 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18606
I20260928 17:58:55.810702 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18681
I20260928 17:58:55.815692 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18756
2026-09-28T17:58:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2838 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:58:55 dist-test-slave-z2t3 krb5kdc[18898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:58:55 dist-test-slave-z2t3 krb5kdc[18898](info): set up 2 sockets
Sep 28 17:58:55 dist-test-slave-z2t3 krb5kdc[18898](info): commencing operation
krb5kdc: starting...
W20260928 17:58:56.860085 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:58:56 dist-test-slave-z2t3 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:58:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:58:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:58:56.983174 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42083
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:57.065769 18911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:57.066023 18911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:57.066061 18911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:57.067247 18911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:57.067323 18911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:57.067348 18911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:57.067368 18911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:57.067400 18911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:57.068933 18911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:57.069231 18911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:57.069471 18911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:57.074837 18916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:57.074844 18917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:57.074865 18911 server_base.cc:1034] running on GCE node
W20260928 17:58:57.074844 18919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:57.075410 18911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:57.075752 18911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:57.076939 18911 hybrid_clock.cc:648] HybridClock initialized: now 1790618337076929 us; error 42 us; skew 500 ppm
Sep 28 17:58:57 dist-test-slave-z2t3 krb5kdc[18898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618337, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:57.079723 18911 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:57.080226 18911 webserver.cc:460] Webserver started at http://127.5.46.126:40399/ using document root <none> and password file <none>
I20260928 17:58:57.080428 18911 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:57.080470 18911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:57.080562 18911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:57.081624 18911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "13fb1bc508fa4917801a8d44e5c9e001"
format_stamp: "Formatted at 2026-09-28 17:58:57 on dist-test-slave-z2t3"
server_key: "a124989b17261ace2852d6a21a1b991a"
server_key_iv: "23a9e9e82e70af06053d19d3a098cdde"
server_key_version: "encryptionkey@0"
I20260928 17:58:57.081920 18911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "13fb1bc508fa4917801a8d44e5c9e001"
format_stamp: "Formatted at 2026-09-28 17:58:57 on dist-test-slave-z2t3"
server_key: "a124989b17261ace2852d6a21a1b991a"
server_key_iv: "23a9e9e82e70af06053d19d3a098cdde"
server_key_version: "encryptionkey@0"
I20260928 17:58:57.083232 18911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:58:57.084028 18926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:57.084226 18911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:58:57.084331 18911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "13fb1bc508fa4917801a8d44e5c9e001"
format_stamp: "Formatted at 2026-09-28 17:58:57 on dist-test-slave-z2t3"
server_key: "a124989b17261ace2852d6a21a1b991a"
server_key_iv: "23a9e9e82e70af06053d19d3a098cdde"
server_key_version: "encryptionkey@0"
I20260928 17:58:57.084421 18911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:57.099699 18911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:57.101966 18911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:57.102154 18911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:57.106197 18911 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:42083
I20260928 17:58:57.106251 18978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:42083 every 8 connection(s)
I20260928 17:58:57.106639 18911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:57.107605 18979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:57.108312 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 18911
I20260928 17:58:57.108433 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:58:57.108641 5305 external_mini_cluster.cc:1428] Setting key 8b0eb2b13d0c30e40278fc883031b330
I20260928 17:58:57.110962 18979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: Bootstrap starting.
I20260928 17:58:57.111706 18979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:57.112072 18979 log.cc:824] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:57.112824 18979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: No bootstrap required, opened a new log
I20260928 17:58:57.115206 18979 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } }
I20260928 17:58:57.115346 18979 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:57.115368 18979 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fb1bc508fa4917801a8d44e5c9e001, State: Initialized, Role: FOLLOWER
I20260928 17:58:57.115516 18979 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } }
I20260928 17:58:57.115682 18979 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:57.115731 18979 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:57.115783 18979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 17:58:57 dist-test-slave-z2t3 krb5kdc[18898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618336, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:57.116720 18979 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } }
I20260928 17:58:57.116844 18979 leader_election.cc:302] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13fb1bc508fa4917801a8d44e5c9e001; no voters:
I20260928 17:58:57.117093 18979 leader_election.cc:288] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:58:57.117120 18984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:57.117233 18984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [term 1 LEADER]: Becoming Leader. State: Replica: 13fb1bc508fa4917801a8d44e5c9e001, State: Running, Role: LEADER
I20260928 17:58:57.117388 18979 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:57.117755 18984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } }
I20260928 17:58:57.118425 18984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13fb1bc508fa4917801a8d44e5c9e001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } } }
I20260928 17:58:57.118532 18984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:57.118682 18984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13fb1bc508fa4917801a8d44e5c9e001. Latest consensus state: current_term: 1 leader_uuid: "13fb1bc508fa4917801a8d44e5c9e001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13fb1bc508fa4917801a8d44e5c9e001" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42083 } } }
I20260928 17:58:57.118763 18984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001 [sys.catalog]: This master's current role is: LEADER
W20260928 17:58:57.119352 18999 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:58:57.119429 18999 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:58:57.119623 18997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:57.119817 18997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:57.121238 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:57.109281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.5.46.126:42083)
0928 17:58:57.109493 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:57.109504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:57.109540 (+ 36us) server_negotiation.cc:407] Connection header received
0928 17:58:57.110160 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:57.110170 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:57.110238 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:57.110411 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:57.111135 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:57.112530 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:57.113356 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:57.113982 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:57.114121 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:57.117009 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:57.117020 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:57.117021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:57.117032 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:57.119494 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:57.119973 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:57.119976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:57.119976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:57.120083 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:57.120333 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:57.120334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:57.120334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:57.120692 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:57.120821 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:57.120982 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 17:58:57.121095 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260928 17:58:57.122288 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 17:58:57.122682 18997 catalog_manager.cc:1347] Generated new cluster ID: 18007bae36444d61a5ba360b73b52bc0
I20260928 17:58:57.122740 18997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:57.144959 18997 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:58:57.145591 18997 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:57.154994 18997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13fb1bc508fa4917801a8d44e5c9e001: Generated new TSK 0
I20260928 17:58:57.155251 18997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 17:58:57.176632 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 18911
2026-09-28T17:58:57Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1359 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:58:57 dist-test-slave-z2t3 krb5kdc[19006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:58:57 dist-test-slave-z2t3 krb5kdc[19006](info): set up 2 sockets
Sep 28 17:58:57 dist-test-slave-z2t3 krb5kdc[19006](info): commencing operation
krb5kdc: starting...
W20260928 17:58:58.220118 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:58:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:58:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:58:58.347915 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:58.432107 19019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:58.432384 19019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:58.432430 19019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:58.433635 19019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:58.433710 19019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:58.433735 19019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:58.433756 19019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:58.433787 19019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:58.435282 19019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:58.435575 19019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:58.435822 19019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:58.441030 19025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.441001 19024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.441037 19019 server_base.cc:1034] running on GCE node
W20260928 17:58:58.440994 19027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.441454 19019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:58.441727 19019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:58.442898 19019 hybrid_clock.cc:648] HybridClock initialized: now 1790618338442886 us; error 36 us; skew 500 ppm
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:58.445518 19019 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:58.446004 19019 webserver.cc:460] Webserver started at http://127.5.46.126:45747/ using document root <none> and password file <none>
I20260928 17:58:58.446259 19019 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:58.446329 19019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:58.446476 19019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:58.447561 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "e4c4bcac469f4e6584af34e81d86f52a"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "68ab249b0b4222db46a15ea4f69de0da"
server_key_iv: "e782dfefa084a4724657873f8100aa4e"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.447980 19019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "e4c4bcac469f4e6584af34e81d86f52a"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "68ab249b0b4222db46a15ea4f69de0da"
server_key_iv: "e782dfefa084a4724657873f8100aa4e"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.449361 19019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:58.450273 19034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.450518 19019 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:58.450654 19019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e4c4bcac469f4e6584af34e81d86f52a"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "68ab249b0b4222db46a15ea4f69de0da"
server_key_iv: "e782dfefa084a4724657873f8100aa4e"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.450805 19019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:58.468189 19019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:58.470897 19019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:58.471076 19019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:58.475785 19019 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:37651
I20260928 17:58:58.475821 19086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:37651 every 8 connection(s)
I20260928 17:58:58.476280 19019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:58.477303 19087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:58.477699 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19019
I20260928 17:58:58.477819 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:58:58.478055 5305 external_mini_cluster.cc:1428] Setting key 42810eb1216808f16c8b748edcb7caf0
I20260928 17:58:58.480293 19087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap starting.
I20260928 17:58:58.480914 19087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:58.481221 19087 log.cc:824] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:58.481959 19087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: No bootstrap required, opened a new log
I20260928 17:58:58.483738 19087 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:58.483951 19087 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:58.484018 19087 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Initialized, Role: FOLLOWER
I20260928 17:58:58.484180 19087 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:58.484275 19087 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:58.484321 19087 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:58.484371 19087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:58.484915 19087 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:58.485039 19087 leader_election.cc:302] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4c4bcac469f4e6584af34e81d86f52a; no voters:
I20260928 17:58:58.485263 19087 leader_election.cc:288] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:58:58.485294 19092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:58.485406 19092 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 LEADER]: Becoming Leader. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Running, Role: LEADER
I20260928 17:58:58.485545 19092 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:58.485545 19087 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:58.486755 19093 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:58:58.486848 19093 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:58.487346 19094 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4c4bcac469f4e6584af34e81d86f52a. Latest consensus state: current_term: 1 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:58:58.487424 19094 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
W20260928 17:58:58.487643 19107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 17:58:58.487720 19107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 17:58:58.488000 19106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:58.488206 19106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:58.489797 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.478734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.5.46.126:37651)
0928 17:58:58.478920 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:58.478931 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:58.479043 (+ 112us) server_negotiation.cc:407] Connection header received
0928 17:58:58.479532 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:58.479539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:58.479591 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:58.479759 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:58.480458 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.481458 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.482299 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.482727 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.482857 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:58.485576 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:58.485581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:58.485581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:58.485599 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:58.487891 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.488322 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.488325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.488326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.488429 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.488623 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.488624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.488624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.488939 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:58.489084 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:58.489422 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 17:58:58.489583 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260928 17:58:58.489801 19106 catalog_manager.cc:1347] Generated new cluster ID: 7468a5537eff4aad9d56fb330e75a987
I20260928 17:58:58.489874 19106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:58.496223 19106 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:58:58.497011 19106 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:58.499440 19106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Generated new TSK 0
I20260928 17:58:58.499620 19106 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:58:58.531545 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:58.616603 19113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:58.616858 19113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:58.616901 19113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:58.618155 19113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:58.618234 19113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:58.618283 19113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:58:58.619908 19113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:58.620201 19113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:58.620458 19113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 17:58:58.625837 19113 server_base.cc:1034] running on GCE node
W20260928 17:58:58.625844 19121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.625815 19119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.625811 19118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.626247 19113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:58.626511 19113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:58.627688 19113 hybrid_clock.cc:648] HybridClock initialized: now 1790618338627649 us; error 58 us; skew 500 ppm
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:58.630239 19113 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:58:58.630745 19113 webserver.cc:460] Webserver started at http://127.5.46.65:38895/ using document root <none> and password file <none>
I20260928 17:58:58.630975 19113 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:58.631019 19113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:58.631177 19113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:58.632345 19113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "6317ab4f41634025b633e604604408dd"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "d84848162f1ddea529876f92ea708b15"
server_key_iv: "7f36b77f700ae9d5a7eef08ce903b8eb"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.632707 19113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "6317ab4f41634025b633e604604408dd"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "d84848162f1ddea529876f92ea708b15"
server_key_iv: "7f36b77f700ae9d5a7eef08ce903b8eb"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.634076 19113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:58:58.634912 19128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.635106 19113 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:58.635223 19113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "6317ab4f41634025b633e604604408dd"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "d84848162f1ddea529876f92ea708b15"
server_key_iv: "7f36b77f700ae9d5a7eef08ce903b8eb"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.635303 19113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:58.646610 19113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:58.648809 19113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:58.648989 19113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:58.649243 19113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:58.649570 19113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:58.649626 19113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.649662 19113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:58.649710 19113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.656039 19113 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36085
I20260928 17:58:58.656083 19241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36085 every 8 connection(s)
I20260928 17:58:58.656481 19113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:58:58.657109 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19113
I20260928 17:58:58.657248 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:58:58.657497 5305 external_mini_cluster.cc:1428] Setting key f262623c0537f48f03ad45b8c05aa13f
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
I20260928 17:58:58.667788 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.657284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:42661 (local address 127.5.46.126:37651)
0928 17:58:58.657339 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:58.657342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:58.657624 (+ 282us) server_negotiation.cc:407] Connection header received
0928 17:58:58.658257 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:58.658258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:58.658270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:58.658320 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:58.659759 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.660629 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.661748 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.662129 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.662210 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:58.664587 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:58.664588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:58.664589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:58.664598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:58.666240 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.666731 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.666733 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.666733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.666796 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.667064 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.667065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.667065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.667224 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:58.667303 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:58.667570 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 17:58:58.667662 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 17:58:58.668042 19244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.657244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:42661)
0928 17:58:58.657455 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:58.657508 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:58.658164 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:58.658419 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:58.658424 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:58.659310 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:58.659610 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:58.659617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.660744 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.660745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:58.661653 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:58.661655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.662231 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.662232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:58.662372 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:58.662977 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:58.662986 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:58.664484 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:58.666334 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:58.666341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:58.666343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:58.666655 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:58.666873 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:58.666874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:58.666874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:58.666980 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:58.667313 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:58.667318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:58.667450 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 17:58:58.667539 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 17:58:58.667705 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260928 17:58:58.668607 19242 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:58.668741 19242 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:58.668964 19242 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
I20260928 17:58:58.669572 19051 ts_manager.cc:194] Registered new tserver with Master: 6317ab4f41634025b633e604604408dd (127.5.46.65:36085)
I20260928 17:58:58.670213 19051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:42661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:58:58.687991 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:58.769503 19247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:58.769768 19247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:58.769809 19247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:58.770982 19247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:58.771060 19247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:58.771106 19247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:58:58.772643 19247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:58.772950 19247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:58.773200 19247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:58.778395 19255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.778391 19252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.778431 19247 server_base.cc:1034] running on GCE node
W20260928 17:58:58.778396 19253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.778918 19247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:58.779173 19247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:58.780335 19247 hybrid_clock.cc:648] HybridClock initialized: now 1790618338780318 us; error 49 us; skew 500 ppm
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:58.782903 19247 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:58:58.783394 19247 webserver.cc:460] Webserver started at http://127.5.46.66:34231/ using document root <none> and password file <none>
I20260928 17:58:58.783651 19247 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:58.783721 19247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:58.783928 19247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:58.785030 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "33094f7b4a7746799d802fb7e5515295"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "308a357c0df612acd7832580a4e48e3f"
server_key_iv: "39fe707ce5828c49481075fe376b1ce2"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.785400 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "33094f7b4a7746799d802fb7e5515295"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "308a357c0df612acd7832580a4e48e3f"
server_key_iv: "39fe707ce5828c49481075fe376b1ce2"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.786952 19247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:58:58.787799 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.788012 19247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:58.788125 19247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "33094f7b4a7746799d802fb7e5515295"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "308a357c0df612acd7832580a4e48e3f"
server_key_iv: "39fe707ce5828c49481075fe376b1ce2"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.788219 19247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:58.823554 19247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:58.825839 19247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:58.826021 19247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:58.826268 19247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:58.826597 19247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:58.826654 19247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.826695 19247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:58.826723 19247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.832963 19247 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33569
I20260928 17:58:58.833011 19375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33569 every 8 connection(s)
I20260928 17:58:58.833352 19247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:58.843240 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.834225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:46713 (local address 127.5.46.126:37651)
0928 17:58:58.834320 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:58.834322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:58.834524 (+ 202us) server_negotiation.cc:407] Connection header received
0928 17:58:58.835124 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:58.835125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:58.835140 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:58.835204 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:58.836247 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.836755 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.837778 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.837992 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.838106 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:58.840540 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:58.840542 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:58.840542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:58.840551 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:58.841815 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.842235 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.842236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.842237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.842307 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:58.842560 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:58.842561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:58.842562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:58.842683 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:58.842760 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:58.843051 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 17:58:58.843147 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 17:58:58.843387 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19247
I20260928 17:58:58.843437 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.834173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:46713)
0928 17:58:58.834389 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:58.834430 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:58.834996 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:58.835287 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:58.835290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:58.835929 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:58.836141 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:58.836145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.836856 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.836857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:58.837682 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:58.837685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.838150 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:58.838151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:58.838270 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:58.838833 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:58.838841 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:58.840440 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:58.841913 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:58.841921 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:58.841924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:58.842157 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:58.842377 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:58.842378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:58.842379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:58.842491 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:58.842764 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:58.842771 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:58.842914 (+ 143us) client_negotiation.cc:769] Sending connection context
0928 17:58:58.843017 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 17:58:58.843185 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":95,"threads_started":1}
I20260928 17:58:58.843492 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:58:58.843710 5305 external_mini_cluster.cc:1428] Setting key 1aa01f5627dc3886fda90faa8ecea415
I20260928 17:58:58.843938 19376 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:58.844036 19376 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:58.844260 19376 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
I20260928 17:58:58.844655 19051 ts_manager.cc:194] Registered new tserver with Master: 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
I20260928 17:58:58.845184 19051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:46713
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:58:58.871523 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:58.955880 19381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:58.956122 19381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:58.956159 19381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:58.957515 19381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:58.957583 19381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:58.957634 19381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:58:58.959128 19381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:58.959410 19381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:58.959671 19381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:58.964823 19387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.964829 19386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:58.964880 19389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:58.965039 19381 server_base.cc:1034] running on GCE node
I20260928 17:58:58.965231 19381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:58.965504 19381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:58.966706 19381 hybrid_clock.cc:648] HybridClock initialized: now 1790618338966668 us; error 51 us; skew 500 ppm
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:58.969254 19381 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:58:58.969689 19381 webserver.cc:460] Webserver started at http://127.5.46.67:42959/ using document root <none> and password file <none>
I20260928 17:58:58.969954 19381 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:58.970027 19381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:58.970185 19381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:58:58.971256 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "efb808fa9dd240d9a2049271fc9a2083"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "daeaf4073bbcdb271857571bb58240bb"
server_key_iv: "e7b3a43062e973b7f4de9bd2b1758d13"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.971611 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "efb808fa9dd240d9a2049271fc9a2083"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "daeaf4073bbcdb271857571bb58240bb"
server_key_iv: "e7b3a43062e973b7f4de9bd2b1758d13"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.973057 19381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:58:58.973863 19396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.974030 19381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:58:58.974143 19381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "efb808fa9dd240d9a2049271fc9a2083"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "daeaf4073bbcdb271857571bb58240bb"
server_key_iv: "e7b3a43062e973b7f4de9bd2b1758d13"
server_key_version: "encryptionkey@0"
I20260928 17:58:58.974218 19381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:58.986327 19381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:58.988766 19381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:58.988940 19381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:58.989192 19381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:58.989542 19381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:58.989598 19381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.989636 19381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:58.989684 19381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:58.995813 19381 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:42043
I20260928 17:58:58.995887 19509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:42043 every 8 connection(s)
I20260928 17:58:58.996304 19381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:58:58.996955 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19381
I20260928 17:58:58.997040 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:58:58.997272 5305 external_mini_cluster.cc:1428] Setting key f0c0de2d1196f10d327d7d319fa86a91
Sep 28 17:58:58 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618338, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:59.006867 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.997072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:60757 (local address 127.5.46.126:37651)
0928 17:58:58.997140 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:58.997142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:58.997380 (+ 238us) server_negotiation.cc:407] Connection header received
0928 17:58:58.998103 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:58.998104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:58.998113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:58.998160 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:58.999215 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:58.999947 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.001073 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.001274 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.001329 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.003771 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.003773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.003774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.003781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.005360 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.005855 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.005856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.005856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.005913 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.006207 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.006208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.006208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.006348 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.006420 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.006719 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.006799 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 17:58:59.007138 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:58.997021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:60757)
0928 17:58:58.997211 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:58.997268 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:58.997981 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:58.998251 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:58.998255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:58.998899 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:58.999106 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:58.999111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.000071 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.000073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.000989 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.000992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.001352 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.001353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.001460 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.002126 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:59.002135 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:59.003649 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:59.005454 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.005461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.005463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.005776 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.005998 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.005999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.005999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.006136 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.006449 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:59.006455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:59.006591 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.006691 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.006838 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260928 17:58:59.007786 19510 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:59.007927 19510 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:59.008087 19510 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
I20260928 17:58:59.008508 19051 ts_manager.cc:194] Registered new tserver with Master: efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043)
I20260928 17:58:59.008972 19051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:60757
I20260928 17:58:59.009253 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:58:59.016950 19090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.010717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.5.46.126:37651)
0928 17:58:59.010760 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.010762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.010798 (+ 36us) server_negotiation.cc:407] Connection header received
0928 17:58:59.010850 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.010851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.010859 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.010915 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:59.011432 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.012024 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.012744 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.012943 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.013036 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.014282 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.014284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.014285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.014292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.015897 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.016213 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.016214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.016214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.016267 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.016427 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.016428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.016428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.016572 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.016699 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.016817 (+ 118us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.016881 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 17:58:59.018584 19051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:58:59.020295 19051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:58:59.027302 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.023175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.5.46.65:36085)
0928 17:58:59.023377 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.023382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.023393 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:58:59.023438 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.023440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.023457 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.023755 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.023868 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.024737 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.026212 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.026846 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.026942 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027007 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.027164 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.027215 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260928 17:58:59.027292 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.022857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36085 (local address 127.0.0.1:44756)
0928 17:58:59.023064 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.023115 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.023187 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.023578 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.023581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.023584 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.023665 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.023667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.024871 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.024872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.026109 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.026112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.026940 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.026941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.027026 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027040 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.027134 (+ 94us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.027201 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":53,"threads_started":1}
I20260928 17:58:59.027668 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.022880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33569 (local address 127.0.0.1:43012)
0928 17:58:59.023375 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.023385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.023443 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.023841 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.023866 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.023869 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.023927 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.023928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.025319 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.025320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.026408 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.026410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.027410 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.027410 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.027483 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027486 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.027547 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.027593 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":41,"thread_start_us":110,"threads_started":1}
I20260928 17:58:59.027720 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.023152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60968 (local address 127.5.46.67:42043)
0928 17:58:59.023347 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.023353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.023695 (+ 342us) server_negotiation.cc:407] Connection header received
0928 17:58:59.023827 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.023830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.023873 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.023930 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.024177 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.025281 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.026472 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.027226 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.027307 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027365 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.027583 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.027636 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":93,"threads_started":1}
I20260928 17:58:59.027721 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.023376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.5.46.66:33569)
0928 17:58:59.023629 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.023634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.023645 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:58:59.023685 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.023687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.023713 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.023794 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.024043 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.025189 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.026524 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.027272 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.027332 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027379 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.027584 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.027635 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":97,"threads_started":1}
I20260928 17:58:59.028033 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.022928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42043 (local address 127.0.0.1:60968)
0928 17:58:59.023560 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.023571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.023727 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.024031 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.024031 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.024033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.024084 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.024085 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.025362 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.025363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.026378 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.026381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.027406 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.027407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.027485 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.027488 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.027929 (+ 441us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.027984 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":7,"thread_start_us":67,"threads_started":1}
I20260928 17:58:59.028815 19444 tablet_service.cc:1467] Processing CreateTablet for tablet 1da31110b86e495a8e16659b19233707 (DEFAULT_TABLE table=test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.028813 19310 tablet_service.cc:1467] Processing CreateTablet for tablet 1da31110b86e495a8e16659b19233707 (DEFAULT_TABLE table=test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.028816 19176 tablet_service.cc:1467] Processing CreateTablet for tablet 1da31110b86e495a8e16659b19233707 (DEFAULT_TABLE table=test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.029189 19444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da31110b86e495a8e16659b19233707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.029191 19310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da31110b86e495a8e16659b19233707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.029191 19176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da31110b86e495a8e16659b19233707. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.031924 19527 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Bootstrap starting.
I20260928 17:58:59.032603 19529 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Bootstrap starting.
I20260928 17:58:59.032913 19527 tablet_bootstrap.cc:654] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.033375 19529 tablet_bootstrap.cc:654] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.033535 19527 log.cc:824] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.033959 19529 log.cc:824] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.034320 19528 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Bootstrap starting.
I20260928 17:58:59.034437 19527 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: No bootstrap required, opened a new log
I20260928 17:58:59.034559 19527 ts_tablet_manager.cc:1397] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:58:59.034725 19529 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: No bootstrap required, opened a new log
I20260928 17:58:59.034796 19529 ts_tablet_manager.cc:1397] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:59.035007 19528 tablet_bootstrap.cc:654] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.035638 19528 log.cc:824] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.036392 19528 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: No bootstrap required, opened a new log
I20260928 17:58:59.036480 19528 ts_tablet_manager.cc:1397] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:59.036767 19527 raft_consensus.cc:348] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.036963 19527 raft_consensus.cc:374] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.037045 19527 raft_consensus.cc:729] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb808fa9dd240d9a2049271fc9a2083, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.036978 19529 raft_consensus.cc:348] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.037182 19527 consensus_queue.cc:260] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.037240 19529 raft_consensus.cc:374] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.037287 19529 raft_consensus.cc:729] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33094f7b4a7746799d802fb7e5515295, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.037480 19527 ts_tablet_manager.cc:1428] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:58:59.037441 19529 consensus_queue.cc:260] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.037678 19510 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
I20260928 17:58:59.037770 19529 ts_tablet_manager.cc:1428] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 17:58:59.037897 19376 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
I20260928 17:58:59.038553 19528 raft_consensus.cc:348] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.038820 19528 raft_consensus.cc:374] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.038887 19528 raft_consensus.cc:729] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6317ab4f41634025b633e604604408dd, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.039079 19528 consensus_queue.cc:260] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.039422 19528 ts_tablet_manager.cc:1428] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:58:59.039454 19242 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
W20260928 17:58:59.084035 19377 tablet.cc:2369] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:58:59.157264 19243 tablet.cc:2369] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 17:58:59.247006 19511 tablet.cc:2369] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:58:59.305850 19535 raft_consensus.cc:482] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:58:59.305984 19535 raft_consensus.cc:504] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.306430 19535 leader_election.cc:288] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569), efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043)
I20260928 17:58:59.309535 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.306746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:55915 (local address 127.5.46.67:42043)
0928 17:58:59.306795 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.306797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.306882 (+ 85us) server_negotiation.cc:407] Connection header received
0928 17:58:59.306947 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.306948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.306962 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.307025 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.307192 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.307674 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.308762 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.309154 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.309253 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.309264 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.309419 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.309456 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 17:58:59.309535 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.306759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:57593 (local address 127.5.46.66:33569)
0928 17:58:59.306805 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.306807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.306820 (+ 13us) server_negotiation.cc:407] Connection header received
0928 17:58:59.306933 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.306934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.306946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.307020 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.307256 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.307739 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.308764 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.309143 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.309240 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.309251 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.309416 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.309463 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 17:58:59.309559 19536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.306635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42043 (local address 127.5.46.65:55915)
0928 17:58:59.306798 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.306809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.306874 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.307030 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.307031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.307033 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.307111 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.307113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.307768 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.307769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.308655 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.308658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.309262 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.309263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.309344 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.309348 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.309414 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.309460 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260928 17:58:59.309571 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.306595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33569 (local address 127.5.46.65:57593)
0928 17:58:59.306763 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.306776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.306840 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.307026 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.307026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.307028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.307175 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.307177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.307826 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.307827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.308655 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.308657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.309243 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.309244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.309325 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.309329 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.309408 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.309488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260928 17:58:59.309949 19464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da31110b86e495a8e16659b19233707" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb808fa9dd240d9a2049271fc9a2083" is_pre_election: true
I20260928 17:58:59.310119 19464 raft_consensus.cc:2454] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 0.
I20260928 17:58:59.310163 19330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da31110b86e495a8e16659b19233707" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33094f7b4a7746799d802fb7e5515295" is_pre_election: true
I20260928 17:58:59.310345 19330 raft_consensus.cc:2454] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 0.
I20260928 17:58:59.310393 19132 leader_election.cc:302] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6317ab4f41634025b633e604604408dd, efb808fa9dd240d9a2049271fc9a2083; no voters:
I20260928 17:58:59.310614 19535 raft_consensus.cc:2781] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:58:59.310688 19535 raft_consensus.cc:482] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:58:59.310720 19535 raft_consensus.cc:3037] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:59.311446 19535 raft_consensus.cc:504] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.311619 19535 leader_election.cc:288] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 election: Requested vote from peers 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569), efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043)
I20260928 17:58:59.311793 19330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da31110b86e495a8e16659b19233707" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33094f7b4a7746799d802fb7e5515295"
I20260928 17:58:59.311813 19464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da31110b86e495a8e16659b19233707" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb808fa9dd240d9a2049271fc9a2083"
I20260928 17:58:59.311930 19330 raft_consensus.cc:3037] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:59.311960 19464 raft_consensus.cc:3037] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:58:59.312645 19330 raft_consensus.cc:2454] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 1.
I20260928 17:58:59.312722 19464 raft_consensus.cc:2454] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 1.
I20260928 17:58:59.312887 19129 leader_election.cc:302] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33094f7b4a7746799d802fb7e5515295, 6317ab4f41634025b633e604604408dd; no voters:
I20260928 17:58:59.313019 19535 raft_consensus.cc:2781] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:58:59.313215 19535 raft_consensus.cc:686] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 LEADER]: Becoming Leader. State: Replica: 6317ab4f41634025b633e604604408dd, State: Running, Role: LEADER
I20260928 17:58:59.313369 19535 consensus_queue.cc:237] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.314411 19051 catalog_manager.cc:5495] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6317ab4f41634025b633e604604408dd (127.5.46.65). New cstate: current_term: 1 leader_uuid: "6317ab4f41634025b633e604604408dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } health_report { overall_health: UNKNOWN } } }
I20260928 17:58:59.385442 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.382667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44760 (local address 127.5.46.65:36085)
0928 17:58:59.382738 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.382740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.382751 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:58:59.382838 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.382839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.382874 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.382965 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:58:59.383138 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.383646 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.384497 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.384704 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.384794 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.385035 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:58:59.385152 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:58:59.385261 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.385282 (+ 21us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.385308 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 17:58:59.387616 19330 raft_consensus.cc:1261] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Refusing update from remote peer 6317ab4f41634025b633e604604408dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:59.387660 19464 raft_consensus.cc:1261] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Refusing update from remote peer 6317ab4f41634025b633e604604408dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:58:59.388038 19535 consensus_queue.cc:1035] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:59.388200 19538 consensus_queue.cc:1035] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:58:59.390559 19541 mvcc.cc:205] Tried to move back new op lower bound from 7334372718128447488 to 7334372717828313088. Current Snapshot: MvccSnapshot[applied={T|T < 7334372718128447488}]
I20260928 17:58:59.393181 19049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:59.393270 19049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:58:59.394191 19049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Sending DeleteTablet for 3 replicas of tablet 1da31110b86e495a8e16659b19233707
I20260928 17:58:59.394497 19310 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da31110b86e495a8e16659b19233707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:59 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:43012
I20260928 17:58:59.394548 19176 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da31110b86e495a8e16659b19233707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:59 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44756
I20260928 17:58:59.394752 19551 tablet_replica.cc:330] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: stopping tablet replica
I20260928 17:58:59.394862 19551 raft_consensus.cc:2229] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:59.394873 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19113
I20260928 17:58:59.394935 19444 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da31110b86e495a8e16659b19233707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:59 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:60968
I20260928 17:58:59.394968 19551 raft_consensus.cc:2258] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:59.395488 19551 ts_tablet_manager.cc:1905] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:59.395792 19553 tablet_replica.cc:330] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: stopping tablet replica
I20260928 17:58:59.395953 19553 raft_consensus.cc:2229] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:59.396063 19553 raft_consensus.cc:2258] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:59.396436 19553 ts_tablet_manager.cc:1905] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:59.397498 19551 ts_tablet_manager.cc:1918] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:59.397578 19551 log.cc:1196] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/1da31110b86e495a8e16659b19233707
I20260928 17:58:59.397857 19551 ts_tablet_manager.cc:1939] T 1da31110b86e495a8e16659b19233707 P 33094f7b4a7746799d802fb7e5515295: Deleting consensus metadata
I20260928 17:58:59.397917 19553 ts_tablet_manager.cc:1918] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:59.397989 19553 log.cc:1196] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/1da31110b86e495a8e16659b19233707
I20260928 17:58:59.398250 19553 ts_tablet_manager.cc:1939] T 1da31110b86e495a8e16659b19233707 P efb808fa9dd240d9a2049271fc9a2083: Deleting consensus metadata
I20260928 17:58:59.398406 19035 catalog_manager.cc:4841] TS 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): tablet 1da31110b86e495a8e16659b19233707 (table test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]) successfully deleted
I20260928 17:58:59.398629 19038 catalog_manager.cc:4841] TS efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): tablet 1da31110b86e495a8e16659b19233707 (table test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]) successfully deleted
W20260928 17:58:59.401170 19037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:36085 (error 108)
I20260928 17:58:59.401348 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19247
I20260928 17:58:59.402122 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.401962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36085 (local address 127.0.0.1:44766)
0928 17:58:59.402034 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.402079 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 17:58:59.402272 19037 catalog_manager.cc:4584] TS 6317ab4f41634025b633e604604408dd (127.5.46.65:36085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1da31110b86e495a8e16659b19233707: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36085: connect: Connection refused (error 111)
I20260928 17:58:59.407124 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19381
I20260928 17:58:59.411693 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19019
I20260928 17:58:59.417255 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--webserver_interface=127.5.46.126
--webserver_port=45747
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:58:59.498607 19555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:59.498813 19555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:59.498836 19555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:59.500025 19555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:58:59.500090 19555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:59.500105 19555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:59.500152 19555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:58:59.500195 19555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:58:59.501688 19555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=45747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:59.501950 19555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:59.502223 19555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:59.512095 19562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.512102 19564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:59.512432 19555 server_base.cc:1034] running on GCE node
W20260928 17:58:59.512095 19561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:59.512820 19555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:59.513167 19555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:59.514334 19555 hybrid_clock.cc:648] HybridClock initialized: now 1790618339514320 us; error 28 us; skew 500 ppm
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:59.523262 19555 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:58:59.523777 19555 webserver.cc:460] Webserver started at http://127.5.46.126:45747/ using document root <none> and password file <none>
I20260928 17:58:59.524065 19555 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:59.524168 19555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:59.525647 19555 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.526461 19571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.526808 19555 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.526925 19555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e4c4bcac469f4e6584af34e81d86f52a"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "68ab249b0b4222db46a15ea4f69de0da"
server_key_iv: "e782dfefa084a4724657873f8100aa4e"
server_key_version: "encryptionkey@0"
I20260928 17:58:59.527369 19555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:59.539988 19555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:59.541988 19555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:59.542188 19555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:59.546578 19555 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:37651
I20260928 17:58:59.546612 19623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:37651 every 8 connection(s)
I20260928 17:58:59.547036 19555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:58:59.548759 19624 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:58:59.549464 19624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap starting.
I20260928 17:58:59.552393 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19555
I20260928 17:58:59.552769 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:36085
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=38895
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:59.557982 19624 log.cc:824] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.560262 19624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:58:59.560595 19624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap complete.
I20260928 17:58:59.563030 19624 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:59.563309 19624 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.563539 19624 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:59.563635 19624 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:58:59.563699 19624 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:58:59.563748 19624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:58:59.564860 19624 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:59.564993 19624 leader_election.cc:302] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4c4bcac469f4e6584af34e81d86f52a; no voters:
I20260928 17:58:59.565201 19624 leader_election.cc:288] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 17:58:59.565287 19628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Leader election won for term 2
I20260928 17:58:59.565481 19628 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 LEADER]: Becoming Leader. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Running, Role: LEADER
I20260928 17:58:59.565485 19624 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:58:59.565924 19628 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:58:59.566239 19629 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:58:59.566318 19629 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:59.566531 19630 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4c4bcac469f4e6584af34e81d86f52a. Latest consensus state: current_term: 2 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:58:59.566618 19630 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
I20260928 17:58:59.566537 19637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:58:59.567462 19637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:58:59.567571 19637 catalog_manager.cc:1259] Loaded cluster ID: 7468a5537eff4aad9d56fb330e75a987
I20260928 17:58:59.567610 19637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:58:59.568117 19637 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 17:58:59.573215 19637 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 17:58:59.576045 19637 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:58:59.576288 19637 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Loaded TSK: 0
I20260928 17:58:59.576508 19637 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 17:58:59.641952 19626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:59.642192 19626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:59.642220 19626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:59.643440 19626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:59.643492 19626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:59.643522 19626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:58:59.645113 19626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:36085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=38895
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:59.645380 19626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:59.645661 19626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:59.651818 19654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.651829 19651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.651822 19652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:59.652079 19626 server_base.cc:1034] running on GCE node
I20260928 17:58:59.652436 19626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:59.652740 19626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:59.653925 19626 hybrid_clock.cc:648] HybridClock initialized: now 1790618339653899 us; error 41 us; skew 500 ppm
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:59.657011 19626 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:58:59.657661 19626 webserver.cc:460] Webserver started at http://127.5.46.65:38895/ using document root <none> and password file <none>
I20260928 17:58:59.657940 19626 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:59.658020 19626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:59.659901 19626 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:58:59.660723 19661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.660934 19626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.661043 19626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "6317ab4f41634025b633e604604408dd"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "d84848162f1ddea529876f92ea708b15"
server_key_iv: "7f36b77f700ae9d5a7eef08ce903b8eb"
server_key_version: "encryptionkey@0"
I20260928 17:58:59.661446 19626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:59.670753 19626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:59.673203 19626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:59.673377 19626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:59.673625 19626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:59.674084 19668 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 17:58:59.676148 19626 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 17:58:59.676234 19626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 17:58:59.676285 19626 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 17:58:59.676945 19626 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 17:58:59.677003 19626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 17:58:59.677073 19668 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Bootstrap starting.
I20260928 17:58:59.684559 19626 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36085
I20260928 17:58:59.684958 19626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:58:59.688235 19668 log.cc:824] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.688396 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19626
I20260928 17:58:59.688810 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:33569
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=34231
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:58:59.690723 19668 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:58:59.691035 19668 tablet_bootstrap.cc:492] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Bootstrap complete.
I20260928 17:58:59.691040 19775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36085 every 8 connection(s)
I20260928 17:58:59.692281 19668 ts_tablet_manager.cc:1397] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20260928 17:58:59.693660 19668 raft_consensus.cc:348] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.693989 19668 raft_consensus.cc:729] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6317ab4f41634025b633e604604408dd, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.694147 19668 consensus_queue.cc:260] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } } peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } }
I20260928 17:58:59.694418 19668 ts_tablet_manager.cc:1428] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:59.703183 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.685719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:46459)
0928 17:58:59.688067 (+ 2348us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.688118 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.688989 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.689853 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.689857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.690344 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:59.690681 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.690688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.692934 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.692935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.693842 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.693845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.695131 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.695132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.695297 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.696049 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:59.696059 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:59.698202 (+ 2143us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:59.701118 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.701126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.701128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.701434 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.701794 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.701794 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.701795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.701928 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.702506 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:59.702511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:59.702675 (+ 164us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.702782 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.702953 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"thread_start_us":100,"threads_started":1}
I20260928 17:58:59.703341 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.685770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:46459 (local address 127.5.46.126:37651)
0928 17:58:59.689267 (+ 3497us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.689278 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.689327 (+ 49us) server_negotiation.cc:407] Connection header received
0928 17:58:59.689480 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.689487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.689549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.689697 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:59.690874 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.692786 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.694125 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.695020 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.695158 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.698342 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.698348 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.698348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.698365 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.701003 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.701545 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.701549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.701549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.701702 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.702010 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.702011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.702012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.702409 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.702520 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.702886 (+ 366us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.703065 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3432,"thread_start_us":84,"threads_started":1}
I20260928 17:58:59.704550 19776 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:59.704684 19776 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:59.704949 19776 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
I20260928 17:58:59.705789 19588 ts_manager.cc:194] Registered new tserver with Master: 6317ab4f41634025b633e604604408dd (127.5.46.65:36085)
I20260928 17:58:59.707188 19588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:46459
I20260928 17:58:59.708488 19776 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
I20260928 17:58:59.712826 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.706498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36085 (local address 127.0.0.1:44770)
0928 17:58:59.706789 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.706822 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.706884 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.709095 (+ 2211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.709098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.709101 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.709178 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.709180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.710634 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.710635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.711537 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.711540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.712588 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.712589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.712650 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.712661 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.712728 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.712769 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":107,"threads_started":1}
I20260928 17:58:59.712934 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.707503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44770 (local address 127.5.46.65:36085)
0928 17:58:59.708832 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.708839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.708851 (+ 12us) server_negotiation.cc:407] Connection header received
0928 17:58:59.708891 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.708894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.708912 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.709003 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.709296 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.710506 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.711624 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.712462 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.712578 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.712645 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.712774 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.712832 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1292,"thread_start_us":111,"threads_started":1}
I20260928 17:58:59.713393 19709 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da31110b86e495a8e16659b19233707 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:58:59 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44770
I20260928 17:58:59.714062 19785 tablet_replica.cc:330] stopping tablet replica
I20260928 17:58:59.714180 19785 raft_consensus.cc:2229] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:58:59.714270 19785 raft_consensus.cc:2258] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:58:59.714560 19785 ts_tablet_manager.cc:1905] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:58:59.716536 19785 ts_tablet_manager.cc:1918] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:58:59.716598 19785 log.cc:1196] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/1da31110b86e495a8e16659b19233707
I20260928 17:58:59.716820 19785 ts_tablet_manager.cc:1939] T 1da31110b86e495a8e16659b19233707 P 6317ab4f41634025b633e604604408dd: Deleting consensus metadata
I20260928 17:58:59.717303 19574 catalog_manager.cc:4841] TS 6317ab4f41634025b633e604604408dd (127.5.46.65:36085): tablet 1da31110b86e495a8e16659b19233707 (table test-table [id=4cfa5a4f618d4206be49a26d8b0198e1]) successfully deleted
W20260928 17:58:59.785820 19781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:59.786108 19781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:59.786152 19781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:59.787499 19781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:59.787595 19781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:59.787647 19781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:58:59.789259 19781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:33569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=34231
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:59.789559 19781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:59.789817 19781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:59.795223 19793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.795243 19791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.795405 19790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:59.795329 19781 server_base.cc:1034] running on GCE node
I20260928 17:58:59.795697 19781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:59.795991 19781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:59.797189 19781 hybrid_clock.cc:648] HybridClock initialized: now 1790618339797175 us; error 44 us; skew 500 ppm
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:59.800043 19781 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:58:59.800529 19781 webserver.cc:460] Webserver started at http://127.5.46.66:34231/ using document root <none> and password file <none>
I20260928 17:58:59.800822 19781 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:59.800913 19781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:59.802354 19781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:58:59.803058 19800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.803273 19781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.803380 19781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "33094f7b4a7746799d802fb7e5515295"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "308a357c0df612acd7832580a4e48e3f"
server_key_iv: "39fe707ce5828c49481075fe376b1ce2"
server_key_version: "encryptionkey@0"
I20260928 17:58:59.803798 19781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:59.817358 19781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:59.819433 19781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:59.819618 19781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:59.819955 19781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:59.820287 19781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:59.820343 19781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.820384 19781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:59.820411 19781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.826462 19781 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33569
I20260928 17:58:59.826506 19913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33569 every 8 connection(s)
I20260928 17:58:59.826911 19781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:58:59.828050 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19781
I20260928 17:58:59.828351 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:42043
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=42959
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:59.837792 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.827689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:50623 (local address 127.5.46.126:37651)
0928 17:58:59.827804 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.827806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.828104 (+ 298us) server_negotiation.cc:407] Connection header received
0928 17:58:59.828979 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.828980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.828991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.829045 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:59.830331 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.830846 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.831936 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.832196 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.832284 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.834842 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.834843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.834844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.834850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.836333 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.836783 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.836785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.836785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.836838 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.837104 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.837105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.837106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.837225 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.837308 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.837588 (+ 280us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.837682 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260928 17:58:59.837992 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.827677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:50623)
0928 17:58:59.827940 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.827994 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.828855 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.829130 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.829135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.829989 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:59.830208 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.830213 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.830960 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.830961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.831787 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.831789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.832308 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.832309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.832441 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.833033 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:59.833041 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:59.834720 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:59.836423 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.836430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.836433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.836668 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.836910 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.836911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.836912 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.837025 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.837299 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:59.837306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:59.837456 (+ 150us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.837569 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.837738 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":192,"threads_started":1}
I20260928 17:58:59.838508 19914 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:59.838605 19914 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:59.838820 19914 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
I20260928 17:58:59.839228 19588 ts_manager.cc:194] Registered new tserver with Master: 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
I20260928 17:58:59.839742 19588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:50623
W20260928 17:58:59.913563 19917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:58:59.913770 19917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:58:59.913796 19917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:58:59.914898 19917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:58:59.914933 19917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:58:59.914959 19917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:58:59.916427 19917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:42043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=42959
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:58:59.916631 19917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:58:59.916810 19917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:58:59.922076 19922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.922080 19923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:58:59.922163 19925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:58:59.922351 19917 server_base.cc:1034] running on GCE node
I20260928 17:58:59.922691 19917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:58:59.922978 19917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:58:59.924124 19917 hybrid_clock.cc:648] HybridClock initialized: now 1790618339924121 us; error 41 us; skew 500 ppm
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:58:59.926836 19917 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:58:59.927362 19917 webserver.cc:460] Webserver started at http://127.5.46.67:42959/ using document root <none> and password file <none>
I20260928 17:58:59.927615 19917 fs_manager.cc:359] Metadata directory not provided
I20260928 17:58:59.927677 19917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:58:59.929152 19917 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.929845 19932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.930083 19917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:58:59.930184 19917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "efb808fa9dd240d9a2049271fc9a2083"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "daeaf4073bbcdb271857571bb58240bb"
server_key_iv: "e7b3a43062e973b7f4de9bd2b1758d13"
server_key_version: "encryptionkey@0"
I20260928 17:58:59.930570 19917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:58:59.945003 19917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:58:59.947422 19917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:58:59.947592 19917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:58:59.947894 19917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:58:59.948237 19917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:58:59.948271 19917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.948323 19917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:58:59.948354 19917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.954243 19917 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:42043
I20260928 17:58:59.954290 20045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:42043 every 8 connection(s)
I20260928 17:58:59.954639 19917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
Sep 28 17:58:59 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618339, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:58:59.964092 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 19917
I20260928 17:58:59.964110 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.955476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:48851 (local address 127.5.46.126:37651)
0928 17:58:59.955561 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.955563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.955779 (+ 216us) server_negotiation.cc:407] Connection header received
0928 17:58:59.956281 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.956282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.956292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.956344 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:59.957279 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.957754 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.958674 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.958866 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.958919 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.961168 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.961170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.961170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.961179 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.962608 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.963071 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.963073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.963073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.963131 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.963391 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.963392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.963393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.963509 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.963586 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.963933 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.964014 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 17:58:59.964316 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.955442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:48851)
0928 17:58:59.955622 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.955658 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.956190 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.956400 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.956403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.956998 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:58:59.957195 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.957199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.957832 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.957833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.958602 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.958603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.958936 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.958936 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.959061 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.959569 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:58:59.959576 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:58:59.961070 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:58:59.962748 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.962754 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.962756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.962978 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.963211 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:58:59.963212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:58:59.963212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:58:59.963323 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:58:59.963578 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:58:59.963584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:58:59.963756 (+ 172us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.963909 (+ 153us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.964073 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260928 17:58:59.964897 20046 heartbeater.cc:347] Connected to a master server at 127.5.46.126:37651
I20260928 17:58:59.965018 20046 heartbeater.cc:464] Registering TS with master...
I20260928 17:58:59.965210 20046 heartbeater.cc:511] Master 127.5.46.126:37651 requested a full tablet report, sending...
I20260928 17:58:59.965787 19588 ts_manager.cc:194] Registered new tserver with Master: efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043)
I20260928 17:58:59.966261 19588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:48851
I20260928 17:58:59.970970 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.964552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52750 (local address 127.5.46.126:37651)
0928 17:58:59.964599 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.964601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.964679 (+ 78us) server_negotiation.cc:407] Connection header received
0928 17:58:59.964785 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.964786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.964797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.964852 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:58:59.965711 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.966290 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.967164 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.967363 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.967439 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.968657 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:58:59.968659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:58:59.968659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:58:59.968666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:58:59.969830 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.970185 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.970186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.970187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.970240 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:58:59.970455 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:58:59.970456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:58:59.970456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:58:59.970570 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:58:59.970767 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.970803 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.970870 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:58:59.971733 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:58:59.974277 19779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.972073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.5.46.126:37651)
0928 17:58:59.972109 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.972111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.972237 (+ 126us) server_negotiation.cc:407] Connection header received
0928 17:58:59.972292 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.972292 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.972319 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.972374 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:58:59.972456 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.972893 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.973616 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.973819 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.973871 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.973989 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:58:59.974074 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:58:59.974128 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.974171 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.974183 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 17:58:59.974586 19588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:58:59.976261 19588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:58:59.978354 19709 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d (DEFAULT_TABLE table=test-table [id=17bc15ea46824e81a673d32cab1df47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.978580 19709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.980237 19668 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Bootstrap starting.
I20260928 17:58:59.980597 19668 tablet_bootstrap.cc:654] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.981156 19668 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: No bootstrap required, opened a new log
I20260928 17:58:59.981212 19668 ts_tablet_manager.cc:1397] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 17:58:59.981324 19668 raft_consensus.cc:348] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.981418 19668 raft_consensus.cc:374] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.981472 19668 raft_consensus.cc:729] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6317ab4f41634025b633e604604408dd, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.981519 19668 consensus_queue.cc:260] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.981609 19668 ts_tablet_manager.cc:1428] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:58:59.982224 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.978065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33569 (local address 127.0.0.1:43026)
0928 17:58:59.978204 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.978216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.978259 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.978638 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.978639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.978641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.978686 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.978687 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.979766 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.979767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.980911 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.980914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.982035 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.982036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.982084 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.982088 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.982130 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.982167 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260928 17:58:59.982318 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.978198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.5.46.66:33569)
0928 17:58:59.978456 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.978459 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.978466 (+ 7us) server_negotiation.cc:407] Connection header received
0928 17:58:59.978522 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.978524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.978540 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.978677 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.978761 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.979661 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.981072 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.981828 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.981895 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.981955 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.982172 (+ 217us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.982218 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":96,"threads_started":1}
I20260928 17:58:59.983513 19848 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d (DEFAULT_TABLE table=test-table [id=17bc15ea46824e81a673d32cab1df47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.983951 19848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.984094 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.978196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.5.46.67:42043)
0928 17:58:59.978381 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 17:58:59.978386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:58:59.980342 (+ 1956us) server_negotiation.cc:407] Connection header received
0928 17:58:59.980389 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:58:59.980392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:58:59.980408 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:58:59.980481 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:58:59.980713 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.981765 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.982868 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.983585 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.983667 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.983733 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:58:59.983969 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 17:58:59.984013 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260928 17:58:59.984090 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:58:59.978065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42043 (local address 127.0.0.1:60984)
0928 17:58:59.980208 (+ 2143us) negotiation.cc:107] Waiting for socket to connect
0928 17:58:59.980220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 17:58:59.980279 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:58:59.980506 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:58:59.980507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:58:59.980510 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:58:59.980557 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.980559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.981905 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.981906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.982795 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:58:59.982797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:58:59.983736 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:58:59.983737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:58:59.983813 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:58:59.983818 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:58:59.983949 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 17:58:59.983989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2119,"mutex_wait_us":24,"thread_start_us":74,"threads_started":1}
I20260928 17:58:59.984822 19980 tablet_service.cc:1467] Processing CreateTablet for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d (DEFAULT_TABLE table=test-table [id=17bc15ea46824e81a673d32cab1df47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:58:59.985121 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:58:59.986763 20055 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Bootstrap starting.
I20260928 17:58:59.987339 20055 tablet_bootstrap.cc:654] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.987725 20056 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Bootstrap starting.
I20260928 17:58:59.987944 20055 log.cc:824] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.988493 20056 tablet_bootstrap.cc:654] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Neither blocks nor log segments found. Creating new log.
I20260928 17:58:59.988755 20055 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: No bootstrap required, opened a new log
I20260928 17:58:59.988811 20055 ts_tablet_manager.cc:1397] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 17:58:59.989080 20056 log.cc:824] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Log is configured to *not* fsync() on all Append() calls
I20260928 17:58:59.989773 20056 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: No bootstrap required, opened a new log
I20260928 17:58:59.989856 20056 ts_tablet_manager.cc:1397] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:58:59.990281 20055 raft_consensus.cc:348] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.990430 20055 raft_consensus.cc:374] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.990453 20055 raft_consensus.cc:729] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33094f7b4a7746799d802fb7e5515295, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.990586 20055 consensus_queue.cc:260] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.990906 19914 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
I20260928 17:58:59.990898 20055 ts_tablet_manager.cc:1428] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 17:58:59.991674 20056 raft_consensus.cc:348] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.991827 20056 raft_consensus.cc:374] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:58:59.991914 20056 raft_consensus.cc:729] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb808fa9dd240d9a2049271fc9a2083, State: Initialized, Role: FOLLOWER
I20260928 17:58:59.992039 20056 consensus_queue.cc:260] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:58:59.992293 20056 ts_tablet_manager.cc:1428] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:58:59.992344 20046 heartbeater.cc:503] Master 127.5.46.126:37651 was elected leader, sending a full tablet report...
W20260928 17:59:00.077517 19915 tablet.cc:2369] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:59:00.130671 19782 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:00.130846 19782 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:00.131247 19782 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
I20260928 17:59:00.134590 20053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.131596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:48729 (local address 127.5.46.67:42043)
0928 17:59:00.131650 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:00.131653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:00.131664 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:59:00.131754 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:00.131755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:00.131769 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:00.131829 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:00.132063 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.132601 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.133861 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.134221 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.134278 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:00.134309 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:00.134469 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 17:59:00.134516 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 17:59:00.134601 19778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.131494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42043 (local address 127.5.46.65:48729)
0928 17:59:00.131571 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.131584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:00.131634 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:00.131894 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:00.131895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:00.131897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:00.131982 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:00.131984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.132689 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.132690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:00.133765 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:00.133769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.134307 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.134308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:00.134390 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:00.134393 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:59:00.134450 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 17:59:00.134509 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 17:59:00.134624 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.131494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33569 (local address 127.5.46.65:51681)
0928 17:59:00.131748 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.131759 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:00.131824 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:00.132014 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:00.132015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:00.132016 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:00.132108 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:00.132110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.132860 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.132861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:00.133699 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:00.133702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.134356 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.134357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:00.134425 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:00.134428 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 17:59:00.134501 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 17:59:00.134533 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":48,"thread_start_us":145,"threads_started":1}
I20260928 17:59:00.134682 20052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.131597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:51681 (local address 127.5.46.66:33569)
0928 17:59:00.131655 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:00.131657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:00.131898 (+ 241us) server_negotiation.cc:407] Connection header received
0928 17:59:00.131930 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:00.131931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:00.131945 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:00.132005 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:00.132197 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.132773 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.133778 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.134230 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.134326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:00.134336 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:00.134551 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 17:59:00.134594 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:59:00.135002 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7cb1b09f6f42b0a4686ec7f082e71d" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb808fa9dd240d9a2049271fc9a2083" is_pre_election: true
I20260928 17:59:00.135077 19868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7cb1b09f6f42b0a4686ec7f082e71d" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33094f7b4a7746799d802fb7e5515295" is_pre_election: true
I20260928 17:59:00.135174 20000 raft_consensus.cc:2454] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 0.
I20260928 17:59:00.135241 19868 raft_consensus.cc:2454] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 0.
I20260928 17:59:00.135427 19665 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6317ab4f41634025b633e604604408dd, efb808fa9dd240d9a2049271fc9a2083; no voters:
I20260928 17:59:00.135548 19782 raft_consensus.cc:2781] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:59:00.135605 19782 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:59:00.135627 19782 raft_consensus.cc:3037] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:00.136370 19782 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:00.136515 19782 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 election: Requested vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
I20260928 17:59:00.136703 19868 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7cb1b09f6f42b0a4686ec7f082e71d" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33094f7b4a7746799d802fb7e5515295"
I20260928 17:59:00.136708 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f7cb1b09f6f42b0a4686ec7f082e71d" candidate_uuid: "6317ab4f41634025b633e604604408dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb808fa9dd240d9a2049271fc9a2083"
I20260928 17:59:00.136795 20000 raft_consensus.cc:3037] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:00.136795 19868 raft_consensus.cc:3037] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:00.137753 20000 raft_consensus.cc:2454] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 1.
I20260928 17:59:00.137753 19868 raft_consensus.cc:2454] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6317ab4f41634025b633e604604408dd in term 1.
I20260928 17:59:00.137929 19665 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6317ab4f41634025b633e604604408dd, efb808fa9dd240d9a2049271fc9a2083; no voters:
I20260928 17:59:00.138020 19782 raft_consensus.cc:2781] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:59:00.138172 19782 raft_consensus.cc:686] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 LEADER]: Becoming Leader. State: Replica: 6317ab4f41634025b633e604604408dd, State: Running, Role: LEADER
I20260928 17:59:00.138273 19782 consensus_queue.cc:237] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:00.139144 19588 catalog_manager.cc:5495] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6317ab4f41634025b633e604604408dd (127.5.46.65). New cstate: current_term: 1 leader_uuid: "6317ab4f41634025b633e604604408dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } health_report { overall_health: HEALTHY } } }
I20260928 17:59:00.169866 19784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.167394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.5.46.65:36085)
0928 17:59:00.167446 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:00.167448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:00.167456 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:00.167535 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:00.167535 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:00.167557 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:00.167620 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:59:00.167731 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.168334 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.169110 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:00.169307 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:00.169367 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:00.169507 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:59:00.169621 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:59:00.169683 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:00.169728 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 17:59:00.169749 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 17:59:00.171686 19868 raft_consensus.cc:1261] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Refusing update from remote peer 6317ab4f41634025b633e604604408dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:00.171684 20000 raft_consensus.cc:1261] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Refusing update from remote peer 6317ab4f41634025b633e604604408dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:00.172158 19782 consensus_queue.cc:1035] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:00.172386 20062 consensus_queue.cc:1035] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:00.174705 20064 mvcc.cc:205] Tried to move back new op lower bound from 7334372721339940864 to 7334372721206943744. Current Snapshot: MvccSnapshot[applied={T|T < 7334372721339940864}]
I20260928 17:59:00.175220 20066 mvcc.cc:205] Tried to move back new op lower bound from 7334372721339940864 to 7334372721206943744. Current Snapshot: MvccSnapshot[applied={T|T < 7334372721339940864}]
I20260928 17:59:00.176945 19588 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:00.177052 19588 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:00.177763 19588 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Sending DeleteTablet for 3 replicas of tablet 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:00.178068 19848 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:00 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:43026
I20260928 17:59:00.178114 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19626
I20260928 17:59:00.178198 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:00 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:60984
I20260928 17:59:00.178728 20074 tablet_replica.cc:330] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: stopping tablet replica
I20260928 17:59:00.178814 20074 raft_consensus.cc:2229] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:00.178889 20074 raft_consensus.cc:2258] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:00.179293 20074 ts_tablet_manager.cc:1905] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:00.180953 20074 ts_tablet_manager.cc:1918] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:00.181051 20074 log.cc:1196] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:00.181303 20074 ts_tablet_manager.cc:1939] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 33094f7b4a7746799d802fb7e5515295: Deleting consensus metadata
I20260928 17:59:00.181747 19572 catalog_manager.cc:4841] TS 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): tablet 2f7cb1b09f6f42b0a4686ec7f082e71d (table test-table [id=17bc15ea46824e81a673d32cab1df47c]) successfully deleted
I20260928 17:59:00.184024 20075 tablet_replica.cc:330] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: stopping tablet replica
I20260928 17:59:00.184203 20075 raft_consensus.cc:2229] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:00.184360 20075 raft_consensus.cc:2258] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 17:59:00.184376 19574 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:36085 (error 108)
I20260928 17:59:00.184898 20075 ts_tablet_manager.cc:1905] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:00.184927 20051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.184763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:36085 (local address 127.0.0.1:44798)
0928 17:59:00.184836 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.184886 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 17:59:00.185060 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19781
W20260928 17:59:00.185047 19574 catalog_manager.cc:4584] TS 6317ab4f41634025b633e604604408dd (127.5.46.65:36085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f7cb1b09f6f42b0a4686ec7f082e71d: Network error: Client connection negotiation failed: client connection to 127.5.46.65:36085: connect: Connection refused (error 111)
I20260928 17:59:00.186854 20075 ts_tablet_manager.cc:1918] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:00.186920 20075 log.cc:1196] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:00.187207 20075 ts_tablet_manager.cc:1939] T 2f7cb1b09f6f42b0a4686ec7f082e71d P efb808fa9dd240d9a2049271fc9a2083: Deleting consensus metadata
I20260928 17:59:00.187669 19575 catalog_manager.cc:4841] TS efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): tablet 2f7cb1b09f6f42b0a4686ec7f082e71d (table test-table [id=17bc15ea46824e81a673d32cab1df47c]) successfully deleted
I20260928 17:59:00.190238 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19917
I20260928 17:59:00.195291 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 19555
I20260928 17:59:00.201166 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--webserver_interface=127.5.46.126
--webserver_port=45747
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:00.286273 20077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:00.286532 20077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:00.286576 20077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:00.287783 20077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:59:00.287859 20077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:00.287930 20077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:00.287959 20077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:59:00.287987 20077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:59:00.289484 20077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:37651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=45747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:00.289786 20077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:00.290025 20077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:00.295157 20082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.295179 20083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.295434 20085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:00.295288 20077 server_base.cc:1034] running on GCE node
I20260928 17:59:00.295691 20077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:00.296080 20077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:00.297226 20077 hybrid_clock.cc:648] HybridClock initialized: now 1790618340297213 us; error 30 us; skew 500 ppm
Sep 28 17:59:00 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:00.299749 20077 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:59:00.300220 20077 webserver.cc:460] Webserver started at http://127.5.46.126:45747/ using document root <none> and password file <none>
I20260928 17:59:00.300448 20077 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:00.300495 20077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:00.301882 20077 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:00.302567 20092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.302771 20077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:00.302878 20077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e4c4bcac469f4e6584af34e81d86f52a"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "68ab249b0b4222db46a15ea4f69de0da"
server_key_iv: "e782dfefa084a4724657873f8100aa4e"
server_key_version: "encryptionkey@0"
I20260928 17:59:00.303260 20077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:00.321733 20077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:00.323971 20077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:00.324158 20077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:00.328109 20077 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:37651
I20260928 17:59:00.328141 20144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:37651 every 8 connection(s)
I20260928 17:59:00.328534 20077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:59:00.330166 20145 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:59:00.330848 20145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap starting.
I20260928 17:59:00.336799 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20077
I20260928 17:59:00.337162 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:36085
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=38895
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:59:00.339946 20145 log.cc:824] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:00.342398 20145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:00.342736 20145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a: Bootstrap complete.
I20260928 17:59:00.345101 20145 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:59:00.345345 20145 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Initialized, Role: FOLLOWER
I20260928 17:59:00.345523 20145 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:59:00.345618 20145 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:59:00.345674 20145 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:59:00.345748 20145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 2 FOLLOWER]: Advancing to term 3
I20260928 17:59:00.346848 20145 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:59:00.346978 20145 leader_election.cc:302] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4c4bcac469f4e6584af34e81d86f52a; no voters:
I20260928 17:59:00.347219 20145 leader_election.cc:288] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 17:59:00.347256 20149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 3 FOLLOWER]: Leader election won for term 3
I20260928 17:59:00.347364 20149 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [term 3 LEADER]: Becoming Leader. State: Replica: e4c4bcac469f4e6584af34e81d86f52a, State: Running, Role: LEADER
I20260928 17:59:00.347505 20145 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:59:00.347481 20149 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } }
I20260928 17:59:00.347883 20150 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:59:00.347983 20150 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:00.348222 20151 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4c4bcac469f4e6584af34e81d86f52a. Latest consensus state: current_term: 3 leader_uuid: "e4c4bcac469f4e6584af34e81d86f52a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4c4bcac469f4e6584af34e81d86f52a" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37651 } } }
I20260928 17:59:00.348327 20151 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4c4bcac469f4e6584af34e81d86f52a [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:00.348464 20154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:59:00.349290 20154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:59:00.349392 20154 catalog_manager.cc:1259] Loaded cluster ID: 7468a5537eff4aad9d56fb330e75a987
I20260928 17:59:00.349689 20154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 17:59:00.350004 20154 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5dfea0c9ad google::LogMessage::Fail() at ??:0
@ 0x7f5dfea0f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5dfea0c4de google::LogMessage::Flush() at ??:0
@ 0x7f5dfea0da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5e069310f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f5e0693a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5e008da0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5e008cf954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5e017336db start_thread at ??:0
@ 0x7f5dfd50a61f clone at ??:0
W20260928 17:59:00.435437 20147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:00.435658 20147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:00.435685 20147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:00.436975 20147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:00.437026 20147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:00.437054 20147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:59:00.438555 20147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:36085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=38895
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:00.438819 20147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:00.439062 20147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:00.444142 20181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:00.444411 20147 server_base.cc:1034] running on GCE node
W20260928 17:59:00.444160 20182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.444149 20184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:00.444761 20147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:00.445039 20147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:00.446202 20147 hybrid_clock.cc:648] HybridClock initialized: now 1790618340446191 us; error 27 us; skew 500 ppm
Sep 28 17:59:00 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:00.448792 20147 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:59:00.449282 20147 webserver.cc:460] Webserver started at http://127.5.46.65:38895/ using document root <none> and password file <none>
I20260928 17:59:00.449532 20147 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:00.449580 20147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:00.450974 20147 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:00.451733 20191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.451958 20147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:00.452059 20147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "6317ab4f41634025b633e604604408dd"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "d84848162f1ddea529876f92ea708b15"
server_key_iv: "7f36b77f700ae9d5a7eef08ce903b8eb"
server_key_version: "encryptionkey@0"
I20260928 17:59:00.452452 20147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:00.466907 20147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:00.469506 20147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:00.469692 20147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:00.469947 20147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:00.470420 20198 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 17:59:00.472410 20147 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 17:59:00.472500 20147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 17:59:00.472550 20147 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 17:59:00.473238 20147 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 17:59:00.473301 20147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 17:59:00.473366 20198 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Bootstrap starting.
I20260928 17:59:00.480618 20147 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:36085
I20260928 17:59:00.480654 20305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:36085 every 8 connection(s)
I20260928 17:59:00.481072 20147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:59:00.483070 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20147
I20260928 17:59:00.483350 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:33569
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=34231
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:59:00.484452 20198 log.cc:824] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:00.487358 20198 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:00.487685 20198 tablet_bootstrap.cc:492] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Bootstrap complete.
I20260928 17:59:00.488941 20198 ts_tablet_manager.cc:1397] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.006s
I20260928 17:59:00.490427 20198 raft_consensus.cc:348] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:00.490767 20198 raft_consensus.cc:729] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6317ab4f41634025b633e604604408dd, State: Initialized, Role: FOLLOWER
I20260928 17:59:00.490916 20198 consensus_queue.cc:260] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:00.491223 20198 ts_tablet_manager.cc:1428] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
W20260928 17:59:00.544162 20308 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:59:00.481741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:60435)
0928 17:59:00.481918 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.481964 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:00.482462 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:00.543916 (+ 61454us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
W20260928 17:59:00.544425 20306 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:37651 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:37651: Client connection negotiation failed: client connection to 127.5.46.126:37651: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 17:59:00.544848 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.544675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:51443)
0928 17:59:00.544759 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.544822 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 17:59:00.545240 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.545149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:43597)
0928 17:59:00.545197 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.545223 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 17:59:00.545368 20306 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:59:00.572803 20309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:00.573028 20309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:00.573055 20309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:00.574276 20309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:00.574322 20309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:00.574349 20309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:59:00.575816 20309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:33569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=34231
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:00.576105 20309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:00.576306 20309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:00.581506 20317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.581548 20316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.581548 20319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:00.582027 20309 server_base.cc:1034] running on GCE node
I20260928 17:59:00.582265 20309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:00.582544 20309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:00.583730 20309 hybrid_clock.cc:648] HybridClock initialized: now 1790618340583717 us; error 31 us; skew 500 ppm
Sep 28 17:59:00 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:00.586282 20309 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:59:00.586769 20309 webserver.cc:460] Webserver started at http://127.5.46.66:34231/ using document root <none> and password file <none>
I20260928 17:59:00.587023 20309 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:00.587093 20309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:00.588533 20309 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:59:00.589159 20326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.589325 20309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:59:00.589428 20309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "33094f7b4a7746799d802fb7e5515295"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "308a357c0df612acd7832580a4e48e3f"
server_key_iv: "39fe707ce5828c49481075fe376b1ce2"
server_key_version: "encryptionkey@0"
I20260928 17:59:00.589799 20309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:00.610512 20309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:00.612810 20309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:00.612989 20309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:00.613245 20309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:00.613574 20309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:00.613631 20309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.613682 20309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:00.613729 20309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.619979 20309 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33569
I20260928 17:59:00.620025 20439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33569 every 8 connection(s)
I20260928 17:59:00.620436 20309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:59:00.621551 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.621181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:38933)
0928 17:59:00.621331 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.621403 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260928 17:59:00.621953 20440 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:37651 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:37651: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
I20260928 17:59:00.622259 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.622163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:45241)
0928 17:59:00.622213 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.622240 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 17:59:00.622548 20442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.622477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:40707)
0928 17:59:00.622515 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.622535 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 17:59:00.622677 20440 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 17:59:00.629390 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20309
I20260928 17:59:00.629643 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:42043
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:37651
--webserver_port=42959
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:00.711870 20443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:00.712141 20443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:00.712182 20443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:00.713392 20443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:00.713471 20443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:00.713518 20443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:59:00.715035 20443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:42043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=42959
--tserver_master_addrs=127.5.46.126:37651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:00.715314 20443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:00.715562 20443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:00.720773 20449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.720813 20451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:00.720785 20448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:00.721035 20443 server_base.cc:1034] running on GCE node
I20260928 17:59:00.721228 20443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:00.721432 20443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:00.722573 20443 hybrid_clock.cc:648] HybridClock initialized: now 1790618340722560 us; error 27 us; skew 500 ppm
Sep 28 17:59:00 dist-test-slave-z2t3 krb5kdc[19006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:00.725334 20443 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:59:00.725775 20443 webserver.cc:460] Webserver started at http://127.5.46.67:42959/ using document root <none> and password file <none>
I20260928 17:59:00.726009 20443 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:00.726058 20443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:00.727397 20443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:00.728040 20458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.728219 20443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:00.728312 20443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "efb808fa9dd240d9a2049271fc9a2083"
format_stamp: "Formatted at 2026-09-28 17:58:58 on dist-test-slave-z2t3"
server_key: "daeaf4073bbcdb271857571bb58240bb"
server_key_iv: "e7b3a43062e973b7f4de9bd2b1758d13"
server_key_version: "encryptionkey@0"
I20260928 17:59:00.728641 20443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:00.747167 20443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:00.749323 20443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:00.749501 20443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:00.749714 20443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:00.750041 20443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:00.750098 20443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.750141 20443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:00.750169 20443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:00.756385 20443 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:42043
I20260928 17:59:00.756482 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:42043 every 8 connection(s)
I20260928 17:59:00.756847 20443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:59:00.758109 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.757673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:49373)
0928 17:59:00.757880 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.757958 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":82,"threads_started":1}
W20260928 17:59:00.758657 20572 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:37651 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:37651: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
I20260928 17:59:00.758987 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.758889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:58777)
0928 17:59:00.758936 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.758966 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 17:59:00.759325 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:00.759209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:43365)
0928 17:59:00.759272 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:00.759302 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260928 17:59:00.759495 20572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 17:59:00.765027 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20443
I20260928 17:59:01.546065 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.545818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:38739)
0928 17:59:01.545952 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:01.546001 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260928 17:59:01.623291 20577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.623066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:36173)
0928 17:59:01.623188 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:01.623232 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260928 17:59:01.760254 20578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.760006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:51079)
0928 17:59:01.760147 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:01.760190 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260928 17:59:01.845760 20579 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:01.845896 20579 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:01.846345 20579 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
Sep 28 17:59:01 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.66@KRBTEST.COM
Sep 28 17:59:01 dist-test-slave-z2t3 krb5kdc[19006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618340, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.67@KRBTEST.COM
I20260928 17:59:01.860210 20582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.846704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:45077 (local address 127.5.46.67:42043)
0928 17:59:01.846841 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:01.846859 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:01.846889 (+ 30us) server_negotiation.cc:407] Connection header received
0928 17:59:01.847431 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:01.847434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:01.847476 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:01.847636 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:01.848717 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.850152 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.851982 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.852631 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.852758 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:01.856550 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:01.856554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:01.856554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:01.856566 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:01.858404 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:01.858892 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:01.858895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:01.858896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:01.858997 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:01.859274 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:01.859275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:01.859275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:01.859514 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:01.859631 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:01.859940 (+ 309us) server_negotiation.cc:299] Negotiation successful
0928 17:59:01.860085 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260928 17:59:01.860198 20576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.846558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:42043 (local address 127.5.46.65:45077)
0928 17:59:01.846622 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:01.846636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:01.846701 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:01.847671 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:01.847674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:01.848223 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:01.848555 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:01.848563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.850309 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.850310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:01.851841 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:01.851873 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.852760 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.852761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:01.852954 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:01.853696 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:01.853708 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:01.856411 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:01.858594 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:01.858595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:01.858595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:01.858791 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:01.859097 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:01.859108 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:01.859108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:01.859194 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:01.859599 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:01.859600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:01.859737 (+ 137us) client_negotiation.cc:769] Sending connection context
0928 17:59:01.859867 (+ 130us) client_negotiation.cc:240] Negotiation successful
0928 17:59:01.860086 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":79}
I20260928 17:59:01.860354 20580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.846578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33569 (local address 127.5.46.65:47127)
0928 17:59:01.846709 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:01.846720 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:01.846779 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:01.847739 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:01.847740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:01.848223 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:01.848555 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:01.848563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.850296 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.850298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:01.851841 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:01.851873 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.852761 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.852762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:01.852954 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:01.853708 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:01.853713 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:01.855735 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:01.858342 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:01.858351 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:01.858354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:01.858708 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:01.859006 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:01.859007 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:01.859008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:01.859134 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:01.859576 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:01.859584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:01.859729 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 17:59:01.859848 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 17:59:01.860174 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":246,"thread_start_us":68,"threads_started":1}
I20260928 17:59:01.860239 20581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:01.846686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:47127 (local address 127.5.46.66:33569)
0928 17:59:01.846894 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:01.846903 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:01.846928 (+ 25us) server_negotiation.cc:407] Connection header received
0928 17:59:01.847383 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:01.847411 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:01.847471 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:01.847693 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:01.848716 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.850151 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.851982 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:01.852685 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:01.852780 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:01.855917 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:01.855922 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:01.855923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:01.855937 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:01.858223 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:01.858815 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:01.858818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:01.858819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:01.858926 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:01.859216 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:01.859217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:01.859217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:01.859487 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:01.859593 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:01.859935 (+ 342us) server_negotiation.cc:299] Negotiation successful
0928 17:59:01.860109 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":66,"threads_started":1}
W20260928 17:59:01.861225 20195 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
W20260928 17:59:01.861373 20192 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:01.861452 20192 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6317ab4f41634025b633e604604408dd; no voters: 33094f7b4a7746799d802fb7e5515295, efb808fa9dd240d9a2049271fc9a2083
I20260928 17:59:01.861635 20579 raft_consensus.cc:2726] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 17:59:02.546972 20583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:02.546687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:49797)
0928 17:59:02.546875 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:02.546925 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260928 17:59:02.624042 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:02.623762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:35279)
0928 17:59:02.623960 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:02.623997 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":63,"threads_started":1}
I20260928 17:59:02.761143 20585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:02.760859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:53299)
0928 17:59:02.761035 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:02.761080 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260928 17:59:03.547765 20586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:03.547536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:35495)
0928 17:59:03.547677 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:03.547716 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260928 17:59:03.624904 20587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:03.624594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:42201)
0928 17:59:03.624788 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:03.624853 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260928 17:59:03.644564 20588 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:03.644650 20588 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:03.644843 20588 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
W20260928 17:59:03.645270 20192 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
W20260928 17:59:03.645347 20195 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:03.645460 20195 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6317ab4f41634025b633e604604408dd; no voters: 33094f7b4a7746799d802fb7e5515295, efb808fa9dd240d9a2049271fc9a2083
I20260928 17:59:03.645543 20588 raft_consensus.cc:2726] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 17:59:03.761916 20589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:03.761651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:56521)
0928 17:59:03.761813 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:03.761858 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260928 17:59:04.548561 20590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:04.548327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:34119)
0928 17:59:04.548462 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:04.548514 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260928 17:59:04.625738 20591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:04.625491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:45819)
0928 17:59:04.625626 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:04.625668 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260928 17:59:04.762776 20592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:04.762511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:34609)
0928 17:59:04.762681 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:04.762728 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
I20260928 17:59:05.506801 20593 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:05.506920 20593 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:05.507104 20593 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
W20260928 17:59:05.507529 20195 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
W20260928 17:59:05.507598 20192 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:05.507666 20192 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6317ab4f41634025b633e604604408dd; no voters: 33094f7b4a7746799d802fb7e5515295, efb808fa9dd240d9a2049271fc9a2083
I20260928 17:59:05.507768 20593 raft_consensus.cc:2726] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 17:59:05.549352 20594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:05.549133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:42803)
0928 17:59:05.549273 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:05.549308 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260928 17:59:05.626540 20595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:05.626290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:44239)
0928 17:59:05.626452 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:05.626495 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
I20260928 17:59:05.763620 20596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:05.763347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:40043)
0928 17:59:05.763525 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:05.763571 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260928 17:59:06.550262 20597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:06.550017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:39497)
0928 17:59:06.550168 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:06.550214 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260928 17:59:06.627302 20598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:06.627053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:35903)
0928 17:59:06.627191 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:06.627234 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260928 17:59:06.764417 20599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:06.764183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:48997)
0928 17:59:06.764329 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:06.764370 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260928 17:59:07.223469 20600 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:07.223577 20600 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:07.223786 20600 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
W20260928 17:59:07.224260 20192 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
W20260928 17:59:07.224361 20195 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:07.224436 20195 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6317ab4f41634025b633e604604408dd; no voters: 33094f7b4a7746799d802fb7e5515295, efb808fa9dd240d9a2049271fc9a2083
I20260928 17:59:07.224521 20600 raft_consensus.cc:2726] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 17:59:07.551213 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:07.550888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:52415)
0928 17:59:07.551052 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:07.551126 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260928 17:59:07.628262 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:07.627994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:59057)
0928 17:59:07.628149 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:07.628189 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":56,"threads_started":1}
I20260928 17:59:07.765297 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:07.765033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:33335)
0928 17:59:07.765202 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:07.765249 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260928 17:59:08.552037 20604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:08.551723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:42931)
0928 17:59:08.551930 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:08.551973 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260928 17:59:08.629032 20605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:08.628778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:44673)
0928 17:59:08.628914 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:08.628966 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260928 17:59:08.766041 20606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:08.765818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:41273)
0928 17:59:08.765962 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:08.765995 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
I20260928 17:59:09.553025 20607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:09.552767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:51397)
0928 17:59:09.552920 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:09.552966 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260928 17:59:09.629873 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:09.629590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:58401)
0928 17:59:09.629764 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:09.629819 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":84,"threads_started":1}
I20260928 17:59:09.766878 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:09.766600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:39203)
0928 17:59:09.766771 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:09.766826 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260928 17:59:10.553944 20610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:10.553654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:54011)
0928 17:59:10.553849 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:10.553897 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260928 17:59:10.630829 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:10.630557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:44321)
0928 17:59:10.630733 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:10.630780 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":87,"threads_started":1}
I20260928 17:59:10.767683 20612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:10.767413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:49501)
0928 17:59:10.767559 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:10.767594 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260928 17:59:11.554836 20613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:11.554539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:47561)
0928 17:59:11.554737 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:11.554786 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":131,"threads_started":1}
I20260928 17:59:11.631558 20614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:11.631345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:43155)
0928 17:59:11.631472 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:11.631508 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260928 17:59:11.768411 20615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:11.768193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:33893)
0928 17:59:11.768326 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:11.768363 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260928 17:59:12.555728 20616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:12.555435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:51867)
0928 17:59:12.555639 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:12.555682 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":80,"threads_started":1}
I20260928 17:59:12.632434 20617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:12.632170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:56453)
0928 17:59:12.632341 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:12.632386 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260928 17:59:12.769244 20618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:12.768955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:41101)
0928 17:59:12.769146 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:12.769193 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
I20260928 17:59:13.028458 20619 raft_consensus.cc:482] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:13.028561 20619 raft_consensus.cc:504] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efb808fa9dd240d9a2049271fc9a2083" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 42043 } } peers { permanent_uuid: "33094f7b4a7746799d802fb7e5515295" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33569 } } peers { permanent_uuid: "6317ab4f41634025b633e604604408dd" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 36085 } }
I20260928 17:59:13.028709 20619 leader_election.cc:288] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043), 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569)
W20260928 17:59:13.029206 20192 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 33094f7b4a7746799d802fb7e5515295 (127.5.46.66:33569): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
W20260928 17:59:13.029281 20195 leader_election.cc:341] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer efb808fa9dd240d9a2049271fc9a2083 (127.5.46.67:42043): Not found: Tablet not found: 2f7cb1b09f6f42b0a4686ec7f082e71d
I20260928 17:59:13.029299 20195 leader_election.cc:302] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6317ab4f41634025b633e604604408dd; no voters: 33094f7b4a7746799d802fb7e5515295, efb808fa9dd240d9a2049271fc9a2083
I20260928 17:59:13.029397 20619 raft_consensus.cc:2726] T 2f7cb1b09f6f42b0a4686ec7f082e71d P 6317ab4f41634025b633e604604408dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 17:59:13.556523 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:13.556247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:35787)
0928 17:59:13.556430 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:13.556477 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260928 17:59:13.633240 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:13.632973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:53409)
0928 17:59:13.633145 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:13.633190 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260928 17:59:13.769961 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:13.769729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:33051)
0928 17:59:13.769871 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:13.769914 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260928 17:59:14.557188 20623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:14.556990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:37973)
0928 17:59:14.557111 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:14.557143 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260928 17:59:14.634088 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:14.633822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:41099)
0928 17:59:14.633995 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:14.634035 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260928 17:59:14.770800 20625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:14.770533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:59725)
0928 17:59:14.770701 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:14.770750 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":72,"threads_started":1}
I20260928 17:59:15.558100 20626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:15.557833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.65:53501)
0928 17:59:15.558002 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:15.558054 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
I20260928 17:59:15.634907 20627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:15.634677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.66:33693)
0928 17:59:15.634828 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:15.634861 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260928 17:59:15.771724 20628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:15.771510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:37651 (local address 127.5.46.67:60133)
0928 17:59:15.771645 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:15.771676 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:37651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260928 17:59:15.776433 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20147
I20260928 17:59:15.782130 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20309
I20260928 17:59:15.786689 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20443
2026-09-28T17:59:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19457 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 17:59:16 dist-test-slave-z2t3 krb5kdc[20631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 17:59:16 dist-test-slave-z2t3 krb5kdc[20631](info): set up 2 sockets
Sep 28 17:59:16 dist-test-slave-z2t3 krb5kdc[20631](info): commencing operation
krb5kdc: starting...
W20260928 17:59:17.672016 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 17:59:17 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T17:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T17:59:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 17:59:17.795138 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:17.878538 20644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:17.878804 20644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:17.878844 20644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:17.880110 20644 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 17:59:17.880169 20644 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 17:59:17.880193 20644 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 17:59:17.880223 20644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:59:17.880272 20644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:17.880306 20644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:17.880333 20644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:59:17.880357 20644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:59:17.881904 20644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:17.882196 20644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:17.882448 20644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:17.887935 20650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:17.888016 20644 server_base.cc:1034] running on GCE node
W20260928 17:59:17.888021 20649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:17.887935 20652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:17.888453 20644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:17.888728 20644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:17.889918 20644 hybrid_clock.cc:648] HybridClock initialized: now 1790618357889902 us; error 46 us; skew 500 ppm
Sep 28 17:59:17 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618357, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:17.892664 20644 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:59:17.893095 20644 webserver.cc:460] Webserver started at http://127.5.46.126:43739/ using document root <none> and password file <none>
I20260928 17:59:17.893325 20644 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:17.893393 20644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:17.893546 20644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:59:17.894622 20644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "a743e415acd44224a1bffec4848d035e"
format_stamp: "Formatted at 2026-09-28 17:59:17 on dist-test-slave-z2t3"
server_key: "e6d6765e5032036ce2dc55b255ad0e8f"
server_key_iv: "6b41590903598e92480f9e4a526a16c0"
server_key_version: "encryptionkey@0"
I20260928 17:59:17.895001 20644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "a743e415acd44224a1bffec4848d035e"
format_stamp: "Formatted at 2026-09-28 17:59:17 on dist-test-slave-z2t3"
server_key: "e6d6765e5032036ce2dc55b255ad0e8f"
server_key_iv: "6b41590903598e92480f9e4a526a16c0"
server_key_version: "encryptionkey@0"
I20260928 17:59:17.896450 20644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:59:17.897265 20659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:17.897600 20644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:17.897707 20644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "a743e415acd44224a1bffec4848d035e"
format_stamp: "Formatted at 2026-09-28 17:59:17 on dist-test-slave-z2t3"
server_key: "e6d6765e5032036ce2dc55b255ad0e8f"
server_key_iv: "6b41590903598e92480f9e4a526a16c0"
server_key_version: "encryptionkey@0"
I20260928 17:59:17.897782 20644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:17.920989 20644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:17.923483 20644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:17.923659 20644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:17.927606 20644 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46515
I20260928 17:59:17.927693 20711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46515 every 8 connection(s)
I20260928 17:59:17.928079 20644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:59:17.928946 20712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:17.930476 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20644
I20260928 17:59:17.930584 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 17:59:17.930783 5305 external_mini_cluster.cc:1428] Setting key ccfc5c747a182946c8f67f987f8724a5
I20260928 17:59:17.931890 20712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap starting.
I20260928 17:59:17.932631 20712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:17.932982 20712 log.cc:824] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:17.933851 20712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: No bootstrap required, opened a new log
I20260928 17:59:17.936285 20712 raft_consensus.cc:348] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:17.936439 20712 raft_consensus.cc:374] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:17.936461 20712 raft_consensus.cc:729] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a743e415acd44224a1bffec4848d035e, State: Initialized, Role: FOLLOWER
I20260928 17:59:17.936685 20712 consensus_queue.cc:260] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:17.936800 20712 raft_consensus.cc:388] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:59:17.936836 20712 raft_consensus.cc:482] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:59:17.936879 20712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:17.937713 20712 raft_consensus.cc:504] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:17.937867 20712 leader_election.cc:302] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a743e415acd44224a1bffec4848d035e; no voters:
I20260928 17:59:17.938069 20712 leader_election.cc:288] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 17:59:17.938119 20717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:59:17.938295 20717 raft_consensus.cc:686] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 LEADER]: Becoming Leader. State: Replica: a743e415acd44224a1bffec4848d035e, State: Running, Role: LEADER
I20260928 17:59:17.938367 20712 sys_catalog.cc:564] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: configured and running, proceeding with master startup.
Sep 28 17:59:17 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:17.938450 20717 consensus_queue.cc:237] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:17.939195 20719 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a743e415acd44224a1bffec4848d035e. Latest consensus state: current_term: 1 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:17.939288 20719 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:17.939525 20726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:59:17.939535 20718 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:17.939641 20718 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:17.940444 20726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:59:17.942071 20726 catalog_manager.cc:1347] Generated new cluster ID: 36e098c1f7a649009327b3410085126c
I20260928 17:59:17.942166 20726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:59:17.943104 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:17.931360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59162 (local address 127.5.46.126:46515)
0928 17:59:17.931528 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:17.931538 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:17.931574 (+ 36us) server_negotiation.cc:407] Connection header received
0928 17:59:17.932131 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:17.932141 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:17.932196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:17.932318 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:17.933162 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:17.934302 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:17.935410 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:17.935821 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:17.935980 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:17.939085 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:17.939089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:17.939090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:17.939103 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:17.941339 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:17.941803 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:17.941806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:17.941807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:17.941919 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:17.942161 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:17.942162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:17.942162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:17.942490 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:17.942659 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:17.942757 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 17:59:17.942905 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260928 17:59:17.953816 20726 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 17:59:17.954561 20726 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:59:17.963450 20726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Generated new TSK 0
I20260928 17:59:17.963686 20726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 17:59:17.993191 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:18.075925 20738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:18.076148 20738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:18.076174 20738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:18.077420 20738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:18.077472 20738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:18.077500 20738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:59:18.079025 20738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:18.079296 20738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:18.079561 20738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:18.084794 20744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:18.084793 20746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:18.084838 20743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:18.085139 20738 server_base.cc:1034] running on GCE node
I20260928 17:59:18.085364 20738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:18.085629 20738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:18.086818 20738 hybrid_clock.cc:648] HybridClock initialized: now 1790618358086799 us; error 49 us; skew 500 ppm
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:18.089622 20738 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:59:18.090108 20738 webserver.cc:460] Webserver started at http://127.5.46.65:44089/ using document root <none> and password file <none>
I20260928 17:59:18.090365 20738 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:18.090440 20738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:18.090607 20738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:59:18.091727 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "900b5f7e24884850b933d54f7b5fcb72"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "5bc56aa4d55804cd2285d7b680870062"
server_key_iv: "63d2057c212b33f3bf77d3db0536c47d"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.092167 20738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "900b5f7e24884850b933d54f7b5fcb72"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "5bc56aa4d55804cd2285d7b680870062"
server_key_iv: "63d2057c212b33f3bf77d3db0536c47d"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.093530 20738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:18.094349 20753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.094518 20738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 17:59:18.094628 20738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "900b5f7e24884850b933d54f7b5fcb72"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "5bc56aa4d55804cd2285d7b680870062"
server_key_iv: "63d2057c212b33f3bf77d3db0536c47d"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.094705 20738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:18.173161 20738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:18.175431 20738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:18.175576 20738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:18.175825 20738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:18.176182 20738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:18.176215 20738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.176237 20738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:18.176250 20738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.182577 20738 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33563
I20260928 17:59:18.182657 20866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33563 every 8 connection(s)
I20260928 17:59:18.183004 20738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:18.189700 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20738
I20260928 17:59:18.189818 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 17:59:18.190085 5305 external_mini_cluster.cc:1428] Setting key 71ef408eff722ee708affd9caaad2a48
I20260928 17:59:18.192934 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.183777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:48371 (local address 127.5.46.126:46515)
0928 17:59:18.183831 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.183833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.184046 (+ 213us) server_negotiation.cc:407] Connection header received
0928 17:59:18.184585 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.184587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.184602 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.184643 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:18.185624 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.186138 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.187156 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.187381 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.187469 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.189721 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:18.189722 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:18.189723 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:18.189731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:18.191407 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.191874 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.191876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.191876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.191942 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.192216 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.192217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.192218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.192346 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:18.192425 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.192731 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.192841 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 17:59:18.193131 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.183717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:48371)
0928 17:59:18.183914 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.183962 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.184470 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.184748 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.184752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.185315 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:18.185514 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.185519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.186241 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.186242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.187061 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.187063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.187482 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.187483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.187611 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.188205 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:18.188214 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:18.189615 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:18.191496 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.191504 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.191508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.191759 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.192014 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.192016 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.192016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.192142 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.192416 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:18.192423 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:18.192579 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.192708 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.192877 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":65,"threads_started":1}
I20260928 17:59:18.193687 20867 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:18.193840 20867 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:18.194056 20867 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:18.194658 20676 ts_manager.cc:194] Registered new tserver with Master: 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563)
I20260928 17:59:18.195314 20676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:48371
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 17:59:18.219213 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:18.301110 20872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:18.301326 20872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:18.301350 20872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:18.302523 20872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:18.302569 20872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:18.302596 20872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:59:18.304132 20872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:18.304416 20872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:18.304699 20872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:18.309845 20878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:18.310025 20872 server_base.cc:1034] running on GCE node
W20260928 17:59:18.309854 20877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:18.309911 20880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:18.310410 20872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:18.310670 20872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:18.311915 20872 hybrid_clock.cc:648] HybridClock initialized: now 1790618358311899 us; error 46 us; skew 500 ppm
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:18.314764 20872 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:59:18.315251 20872 webserver.cc:460] Webserver started at http://127.5.46.66:39239/ using document root <none> and password file <none>
I20260928 17:59:18.315503 20872 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:18.315575 20872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:18.315759 20872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:59:18.316926 20872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "2a61b8261dbf821a9b00388a582f9564"
server_key_iv: "3677d4f276c2b4a49e866e03b79d3524"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.317299 20872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "2a61b8261dbf821a9b00388a582f9564"
server_key_iv: "3677d4f276c2b4a49e866e03b79d3524"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.318727 20872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 17:59:18.319515 20887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.319752 20872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:18.319890 20872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "2a61b8261dbf821a9b00388a582f9564"
server_key_iv: "3677d4f276c2b4a49e866e03b79d3524"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.319978 20872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:18.327488 20872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:18.329569 20872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:18.329744 20872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:18.329995 20872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:18.330314 20872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:18.330368 20872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.330411 20872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:18.330436 20872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.336694 20872 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:41179
I20260928 17:59:18.336745 21000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:41179 every 8 connection(s)
I20260928 17:59:18.337061 20872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:18.344385 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 20872
I20260928 17:59:18.344498 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 17:59:18.344729 5305 external_mini_cluster.cc:1428] Setting key 004b920c3795a830b12a12a07205bf4e
I20260928 17:59:18.347436 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.337752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:58093 (local address 127.5.46.126:46515)
0928 17:59:18.337795 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.337797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.338052 (+ 255us) server_negotiation.cc:407] Connection header received
0928 17:59:18.338629 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.338631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.338646 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.338697 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:18.340105 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.340617 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.341565 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.341748 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.341801 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.344065 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:18.344066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:18.344066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:18.344072 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:18.345778 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.346307 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.346308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.346309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.346381 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.346712 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.346714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.346714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.346881 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:18.346941 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.347305 (+ 364us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.347374 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 17:59:18.347765 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.337764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:58093)
0928 17:59:18.337922 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.337958 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.338476 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.338883 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.338889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.339696 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:18.340013 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.340018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.340699 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.340700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.341494 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.341496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.341819 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.341820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.341919 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.342494 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:18.342503 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:18.343978 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:18.345884 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.345892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.345894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.346204 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.346466 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.346467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.346468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.346623 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.346954 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:18.346960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:18.347152 (+ 192us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.347288 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.347477 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260928 17:59:18.348327 21001 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:18.348464 21001 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:18.348673 21001 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:18.349085 20676 ts_manager.cc:194] Registered new tserver with Master: 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
I20260928 17:59:18.349462 20676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:58093
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 17:59:18.371592 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:18.454129 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:18.454340 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:18.454363 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:18.455569 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:18.455621 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:18.455665 21006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:59:18.457975 21006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:18.458359 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:18.458617 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:18.465852 21012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:18.466463 21014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:18.465853 21011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:18.467073 21006 server_base.cc:1034] running on GCE node
I20260928 17:59:18.467358 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:18.467661 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:18.468844 21006 hybrid_clock.cc:648] HybridClock initialized: now 1790618358468813 us; error 50 us; skew 500 ppm
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:18.471478 21006 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:59:18.471935 21006 webserver.cc:460] Webserver started at http://127.5.46.67:39439/ using document root <none> and password file <none>
I20260928 17:59:18.472153 21006 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:18.472215 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:18.472308 21006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 17:59:18.473321 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "f7bc3b1cadd240adb2119d7329736499"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "1064683d67b859bc4a4b1596c2971f90"
server_key_iv: "79850329c64fd34e437df6e92443f63e"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.473624 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "f7bc3b1cadd240adb2119d7329736499"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "1064683d67b859bc4a4b1596c2971f90"
server_key_iv: "79850329c64fd34e437df6e92443f63e"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.474932 21006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 17:59:18.475575 21021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.475759 21006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 17:59:18.475833 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "f7bc3b1cadd240adb2119d7329736499"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "1064683d67b859bc4a4b1596c2971f90"
server_key_iv: "79850329c64fd34e437df6e92443f63e"
server_key_version: "encryptionkey@0"
I20260928 17:59:18.475952 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:18.493098 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:18.495420 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:18.495599 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:18.495934 21006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:18.496312 21006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:18.496353 21006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.496378 21006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:18.496424 21006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:18.502815 21006 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:37417
I20260928 17:59:18.502856 21134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:37417 every 8 connection(s)
I20260928 17:59:18.503183 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:59:18.506568 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21006
I20260928 17:59:18.506681 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 17:59:18.506840 5305 external_mini_cluster.cc:1428] Setting key 3a4e42174d92739660613fbce8bd35ba
Sep 28 17:59:18 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618358, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:18.513039 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.503989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:54229 (local address 127.5.46.126:46515)
0928 17:59:18.504050 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.504053 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.504262 (+ 209us) server_negotiation.cc:407] Connection header received
0928 17:59:18.504935 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.504936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.504950 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.505013 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:18.505938 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.506512 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.507774 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.508090 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.508147 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.510325 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:18.510326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:18.510326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:18.510332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:18.511601 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.512034 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.512035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.512035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.512090 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.512301 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.512301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.512302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.512420 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:18.512498 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.512900 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.512972 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 17:59:18.513209 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.503948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:54229)
0928 17:59:18.504130 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.504183 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.504804 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.505077 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.505080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.505664 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:18.505859 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.505864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.506605 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.506605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.507678 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.507681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.508160 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.508161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.508284 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.508799 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:18.508807 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:18.510218 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:18.511672 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.511677 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.511679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.511968 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.512156 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:18.512157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:18.512157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:18.512260 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:18.512578 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:18.512584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:18.512745 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.512845 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.512976 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
I20260928 17:59:18.513760 21135 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:18.513871 21135 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:18.514050 21135 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:18.514458 20676 ts_manager.cc:194] Registered new tserver with Master: f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417)
I20260928 17:59:18.514808 20676 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:54229
I20260928 17:59:18.518782 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:59:18.526079 20715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.520034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59170 (local address 127.5.46.126:46515)
0928 17:59:18.520124 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.520126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.520283 (+ 157us) server_negotiation.cc:407] Connection header received
0928 17:59:18.520310 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.520310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.520321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.520386 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:18.520946 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.521453 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.522175 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.522365 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.522419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.523602 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:18.523604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:18.523604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:18.523611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:18.525180 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.525468 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.525469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.525469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.525511 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:18.525620 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:18.525621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:18.525621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:18.525758 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:18.525834 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.525921 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.525980 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 17:59:18.527705 20676 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:59:18.529372 20676 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:59:18.536882 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.531976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.5.46.67:37417)
0928 17:59:18.532410 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.532420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.532428 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:18.532481 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.532483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.532499 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.532773 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:18.532843 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.533889 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.535542 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.536319 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.536395 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.536452 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.536758 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.536805 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":102,"threads_started":1}
I20260928 17:59:18.536945 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.531930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58236 (local address 127.5.46.65:33563)
0928 17:59:18.532167 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.532180 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.532247 (+ 67us) server_negotiation.cc:407] Connection header received
0928 17:59:18.532386 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.532389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.532404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.532524 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:18.532708 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.533868 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.535934 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.536464 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.536539 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.536598 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.536802 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.536848 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260928 17:59:18.537097 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.531836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37417 (local address 127.0.0.1:48090)
0928 17:59:18.532157 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.532208 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.532271 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.532659 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.532660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.532662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:18.532720 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.532724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.533996 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.533997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.535454 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.535456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.536411 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.536412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.536495 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.536512 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.536582 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.536642 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":103,"thread_start_us":124,"threads_started":1}
I20260928 17:59:18.537114 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.531790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33563 (local address 127.0.0.1:58236)
0928 17:59:18.532152 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.532197 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.532269 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.532531 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.532534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.532537 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:18.532616 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.532619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.533996 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.533997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.535453 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.535456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.536538 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.536540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.536608 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.536613 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.536665 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.536705 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":44,"thread_start_us":93,"threads_started":1}
I20260928 17:59:18.537794 20801 tablet_service.cc:1467] Processing CreateTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f (DEFAULT_TABLE table=test-table [id=b7444720a21d41d49727dec7ff82a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:18.537871 21069 tablet_service.cc:1467] Processing CreateTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f (DEFAULT_TABLE table=test-table [id=b7444720a21d41d49727dec7ff82a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:18.538123 21147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.532025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41179 (local address 127.0.0.1:60270)
0928 17:59:18.534027 (+ 2002us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.534037 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.534118 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.534308 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.534309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.534311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:18.534377 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.534379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.535366 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.535367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.536593 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.536595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.537871 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.537872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.537933 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.537938 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.538007 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.538048 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1982,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260928 17:59:18.538146 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.532195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60270 (local address 127.5.46.66:41179)
0928 17:59:18.532358 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.532362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.534118 (+ 1756us) server_negotiation.cc:407] Connection header received
0928 17:59:18.534198 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.534201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.534216 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.534311 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:18.534435 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.535263 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.536669 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.537773 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.537856 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.537914 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.538022 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.538064 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260928 17:59:18.538214 21069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f2441bb7374fa58c0f5ecb1a557f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:18.538242 20801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f2441bb7374fa58c0f5ecb1a557f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:18.538842 20935 tablet_service.cc:1467] Processing CreateTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f (DEFAULT_TABLE table=test-table [id=b7444720a21d41d49727dec7ff82a8d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:18.539168 20935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54f2441bb7374fa58c0f5ecb1a557f1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:18.541229 21152 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Bootstrap starting.
I20260928 17:59:18.541627 21153 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap starting.
I20260928 17:59:18.542129 21152 tablet_bootstrap.cc:654] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:18.542276 21153 tablet_bootstrap.cc:654] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:18.542492 21154 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Bootstrap starting.
I20260928 17:59:18.542938 21153 log.cc:824] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:18.542938 21152 log.cc:824] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:18.543280 21154 tablet_bootstrap.cc:654] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:18.543943 21154 log.cc:824] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:18.544046 21153 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: No bootstrap required, opened a new log
I20260928 17:59:18.544061 21152 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: No bootstrap required, opened a new log
I20260928 17:59:18.544122 21153 ts_tablet_manager.cc:1397] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:59:18.544126 21152 ts_tablet_manager.cc:1397] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 17:59:18.544704 21154 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: No bootstrap required, opened a new log
I20260928 17:59:18.544777 21154 ts_tablet_manager.cc:1397] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 17:59:18.545640 21152 raft_consensus.cc:348] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.545825 21152 raft_consensus.cc:374] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:18.546051 21152 raft_consensus.cc:729] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7bc3b1cadd240adb2119d7329736499, State: Initialized, Role: FOLLOWER
I20260928 17:59:18.546196 21152 consensus_queue.cc:260] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.546166 21153 raft_consensus.cc:348] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.546351 21153 raft_consensus.cc:374] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:18.546387 21153 raft_consensus.cc:729] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900b5f7e24884850b933d54f7b5fcb72, State: Initialized, Role: FOLLOWER
I20260928 17:59:18.546509 21152 ts_tablet_manager.cc:1428] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 17:59:18.546516 21153 consensus_queue.cc:260] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.546617 21135 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
I20260928 17:59:18.546808 21153 ts_tablet_manager.cc:1428] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 17:59:18.546844 20867 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
I20260928 17:59:18.546840 21154 raft_consensus.cc:348] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.547019 21154 raft_consensus.cc:374] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:18.547073 21154 raft_consensus.cc:729] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86fa9353a3f7497ab8ef9ea317a3543d, State: Initialized, Role: FOLLOWER
I20260928 17:59:18.547186 21154 consensus_queue.cc:260] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.547444 21154 ts_tablet_manager.cc:1428] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 17:59:18.547466 21001 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
W20260928 17:59:18.587623 21002 tablet.cc:2369] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:59:18.670054 21160 raft_consensus.cc:482] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:18.670192 21160 raft_consensus.cc:504] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.670673 21160 leader_election.cc:288] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563), f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417)
I20260928 17:59:18.673625 21003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.670904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37417 (local address 127.5.46.66:46595)
0928 17:59:18.670960 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.670973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.671039 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.671254 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.671256 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.671258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:18.671338 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.671340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.672004 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.672005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.672783 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.672785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.673363 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.673364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.673443 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.673445 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.673511 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.673552 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 17:59:18.673625 21149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.671043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:46595 (local address 127.5.46.67:37417)
0928 17:59:18.671083 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.671085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.671096 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:59:18.671128 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.671129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.671137 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.671240 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:18.671419 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.671909 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.672883 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.673270 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.673350 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.673363 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.673516 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.673553 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 17:59:18.673962 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.670999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:57549 (local address 127.5.46.65:33563)
0928 17:59:18.671050 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.671052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.671266 (+ 214us) server_negotiation.cc:407] Connection header received
0928 17:59:18.671320 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.671321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.671334 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.671403 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:18.671651 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.672232 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.673171 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.673539 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.673625 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.673636 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.673852 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.673895 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 17:59:18.674003 21089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f2441bb7374fa58c0f5ecb1a557f1f" candidate_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bc3b1cadd240adb2119d7329736499" is_pre_election: true
I20260928 17:59:18.674165 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.670905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33563 (local address 127.5.46.66:57549)
0928 17:59:18.671167 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.671183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:18.671264 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:18.671419 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:18.671420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:18.671421 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:18.671500 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.671501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.672328 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.672329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.673071 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:18.673073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.673662 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.673663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:18.673752 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.673757 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:59:18.674054 (+ 297us) client_negotiation.cc:240] Negotiation successful
0928 17:59:18.674095 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":81,"thread_start_us":133,"threads_started":1}
I20260928 17:59:18.674176 21089 raft_consensus.cc:2454] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86fa9353a3f7497ab8ef9ea317a3543d in term 0.
I20260928 17:59:18.674453 20888 leader_election.cc:302] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499; no voters:
I20260928 17:59:18.674494 20821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f2441bb7374fa58c0f5ecb1a557f1f" candidate_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900b5f7e24884850b933d54f7b5fcb72" is_pre_election: true
I20260928 17:59:18.674603 21160 raft_consensus.cc:2781] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:59:18.674652 20821 raft_consensus.cc:2454] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86fa9353a3f7497ab8ef9ea317a3543d in term 0.
I20260928 17:59:18.674680 21160 raft_consensus.cc:482] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:59:18.674710 21160 raft_consensus.cc:3037] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:18.675498 21160 raft_consensus.cc:504] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.675655 21160 leader_election.cc:288] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [CANDIDATE]: Term 1 election: Requested vote from peers 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563), f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417)
I20260928 17:59:18.675817 21089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f2441bb7374fa58c0f5ecb1a557f1f" candidate_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bc3b1cadd240adb2119d7329736499"
I20260928 17:59:18.675832 20821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54f2441bb7374fa58c0f5ecb1a557f1f" candidate_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "900b5f7e24884850b933d54f7b5fcb72"
I20260928 17:59:18.675944 20821 raft_consensus.cc:3037] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:18.675936 21089 raft_consensus.cc:3037] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:18.676676 20821 raft_consensus.cc:2454] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86fa9353a3f7497ab8ef9ea317a3543d in term 1.
I20260928 17:59:18.676759 21089 raft_consensus.cc:2454] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86fa9353a3f7497ab8ef9ea317a3543d in term 1.
I20260928 17:59:18.676854 20890 leader_election.cc:302] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86fa9353a3f7497ab8ef9ea317a3543d, 900b5f7e24884850b933d54f7b5fcb72; no voters:
I20260928 17:59:18.676973 21160 raft_consensus.cc:2781] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:59:18.677160 21160 raft_consensus.cc:686] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 LEADER]: Becoming Leader. State: Replica: 86fa9353a3f7497ab8ef9ea317a3543d, State: Running, Role: LEADER
I20260928 17:59:18.677239 21160 consensus_queue.cc:237] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:18.678117 20676 catalog_manager.cc:5495] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d reported cstate change: term changed from 0 to 1, leader changed from <none> to 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66). New cstate: current_term: 1 leader_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } health_report { overall_health: UNKNOWN } } }
I20260928 17:59:18.683319 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.681205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.5.46.66:41179)
0928 17:59:18.681247 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:18.681253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:18.681261 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:18.681310 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:18.681310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:18.681325 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:18.681381 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:59:18.681482 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.681939 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.682729 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:18.682902 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:18.682954 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:18.683073 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:59:18.683160 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:59:18.683219 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:18.683250 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 17:59:18.683266 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 17:59:18.683781 20868 tablet.cc:2369] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:59:18.685257 20821 raft_consensus.cc:1261] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Refusing update from remote peer 86fa9353a3f7497ab8ef9ea317a3543d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:18.685297 21089 raft_consensus.cc:1261] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Refusing update from remote peer 86fa9353a3f7497ab8ef9ea317a3543d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:18.685655 21160 consensus_queue.cc:1035] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [LEADER]: Connected to new peer: Peer: permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:18.685818 21162 consensus_queue.cc:1035] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:18.688637 21166 mvcc.cc:205] Tried to move back new op lower bound from 7334372797172109312 to 7334372797142560768. Current Snapshot: MvccSnapshot[applied={T|T < 7334372797172109312}]
I20260928 17:59:18.689210 21167 mvcc.cc:205] Tried to move back new op lower bound from 7334372797172109312 to 7334372797142560768. Current Snapshot: MvccSnapshot[applied={T|T < 7334372797172109312}]
I20260928 17:59:18.690430 20676 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:18.690534 20676 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:18.691349 20676 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Sending DeleteTablet for 3 replicas of tablet 54f2441bb7374fa58c0f5ecb1a557f1f
I20260928 17:59:18.691685 20801 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:18 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:58236
I20260928 17:59:18.691776 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20738
I20260928 17:59:18.691777 20935 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:18 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:60270
I20260928 17:59:18.692319 21069 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:18 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:48090
I20260928 17:59:18.693145 21176 tablet_replica.cc:330] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: stopping tablet replica
I20260928 17:59:18.693249 21176 raft_consensus.cc:2229] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:59:18.693425 21177 tablet_replica.cc:330] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: stopping tablet replica
I20260928 17:59:18.693463 21176 raft_consensus.cc:2258] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:18.693558 21177 raft_consensus.cc:2229] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:18.693711 21177 raft_consensus.cc:2258] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:18.694156 21177 ts_tablet_manager.cc:1905] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:18.694161 21176 ts_tablet_manager.cc:1905] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:18.696151 21177 ts_tablet_manager.cc:1918] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:18.696266 21177 log.cc:1196] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/54f2441bb7374fa58c0f5ecb1a557f1f
I20260928 17:59:18.696365 21176 ts_tablet_manager.cc:1918] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:18.696501 21176 log.cc:1196] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/54f2441bb7374fa58c0f5ecb1a557f1f
I20260928 17:59:18.696591 21177 ts_tablet_manager.cc:1939] T 54f2441bb7374fa58c0f5ecb1a557f1f P f7bc3b1cadd240adb2119d7329736499: Deleting consensus metadata
I20260928 17:59:18.696866 21176 ts_tablet_manager.cc:1939] T 54f2441bb7374fa58c0f5ecb1a557f1f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting consensus metadata
I20260928 17:59:18.697105 20660 catalog_manager.cc:4841] TS f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): tablet 54f2441bb7374fa58c0f5ecb1a557f1f (table test-table [id=b7444720a21d41d49727dec7ff82a8d4]) successfully deleted
I20260928 17:59:18.697417 20662 catalog_manager.cc:4841] TS 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): tablet 54f2441bb7374fa58c0f5ecb1a557f1f (table test-table [id=b7444720a21d41d49727dec7ff82a8d4]) successfully deleted
W20260928 17:59:18.698036 20662 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:33563 (error 108)
I20260928 17:59:18.698163 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20872
I20260928 17:59:18.700129 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:18.699981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33563 (local address 127.0.0.1:58248)
0928 17:59:18.700024 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:18.700073 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260928 17:59:18.700330 20662 catalog_manager.cc:4584] TS 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54f2441bb7374fa58c0f5ecb1a557f1f: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33563: connect: Connection refused (error 111)
I20260928 17:59:18.703440 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21006
I20260928 17:59:18.708696 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 20644
I20260928 17:59:38.714237 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--webserver_interface=127.5.46.126
--webserver_port=43739
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:38.795919 21179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:38.796159 21179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:38.796185 21179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:38.797441 21179 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 17:59:38.797489 21179 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 17:59:38.797503 21179 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 17:59:38.797515 21179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:59:38.797528 21179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:38.797541 21179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:38.797554 21179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:59:38.797616 21179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:59:38.799249 21179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=43739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:38.799522 21179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:38.799788 21179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:38.809022 21188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:38.808981 21186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:38.808993 21185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:38.809041 21179 server_base.cc:1034] running on GCE node
I20260928 17:59:38.809577 21179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:38.809937 21179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:38.811105 21179 hybrid_clock.cc:648] HybridClock initialized: now 1790618378811068 us; error 65 us; skew 500 ppm
Sep 28 17:59:38 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:38.814039 21179 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:59:38.814527 21179 webserver.cc:460] Webserver started at http://127.5.46.126:43739/ using document root <none> and password file <none>
I20260928 17:59:38.814828 21179 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:38.814934 21179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:38.816468 21179 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 17:59:38.817214 21195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:38.817466 21179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:38.817593 21179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "a743e415acd44224a1bffec4848d035e"
format_stamp: "Formatted at 2026-09-28 17:59:17 on dist-test-slave-z2t3"
server_key: "e6d6765e5032036ce2dc55b255ad0e8f"
server_key_iv: "6b41590903598e92480f9e4a526a16c0"
server_key_version: "encryptionkey@0"
I20260928 17:59:38.818066 21179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:38.830423 21179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:38.833036 21179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:38.833254 21179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:38.837491 21179 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46515
I20260928 17:59:38.837586 21247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46515 every 8 connection(s)
I20260928 17:59:38.837996 21179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:59:38.839480 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21179
I20260928 17:59:38.839783 21248 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:59:38.839799 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:33563
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=44089
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:59:38.840602 21248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap starting.
I20260928 17:59:38.850258 21248 log.cc:824] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:38.852567 21248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:38.852912 21248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap complete.
I20260928 17:59:38.855499 21248 raft_consensus.cc:348] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:38.855784 21248 raft_consensus.cc:729] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a743e415acd44224a1bffec4848d035e, State: Initialized, Role: FOLLOWER
I20260928 17:59:38.856050 21248 consensus_queue.cc:260] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:38.856170 21248 raft_consensus.cc:388] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:59:38.856240 21248 raft_consensus.cc:482] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:59:38.856295 21248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 1 FOLLOWER]: Advancing to term 2
I20260928 17:59:38.857708 21248 raft_consensus.cc:504] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:38.857848 21248 leader_election.cc:302] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a743e415acd44224a1bffec4848d035e; no voters:
I20260928 17:59:38.858050 21248 leader_election.cc:288] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 17:59:38.858206 21252 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Leader election won for term 2
I20260928 17:59:38.858340 21252 raft_consensus.cc:686] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 LEADER]: Becoming Leader. State: Replica: a743e415acd44224a1bffec4848d035e, State: Running, Role: LEADER
I20260928 17:59:38.858364 21248 sys_catalog.cc:564] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:59:38.858440 21252 consensus_queue.cc:237] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:38.858817 21253 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:38.858903 21253 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:38.859138 21259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:59:38.859166 21254 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a743e415acd44224a1bffec4848d035e. Latest consensus state: current_term: 2 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:38.859915 21254 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:38.860054 21259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:59:38.860157 21259 catalog_manager.cc:1259] Loaded cluster ID: 36e098c1f7a649009327b3410085126c
I20260928 17:59:38.860199 21259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:59:38.861500 21259 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:59:38.861685 21259 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Loaded TSK: 0
I20260928 17:59:38.873457 21259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Generated new TSK 1
I20260928 17:59:38.873580 21259 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 17:59:38.928063 21250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:38.928278 21250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:38.928304 21250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:38.929544 21250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:38.929594 21250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:38.929623 21250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:59:38.931138 21250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:33563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=44089
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:38.931393 21250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:38.931663 21250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:38.936842 21278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:38.936842 21275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:38.937119 21250 server_base.cc:1034] running on GCE node
W20260928 17:59:38.937202 21276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:38.937453 21250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:38.937739 21250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:38.938895 21250 hybrid_clock.cc:648] HybridClock initialized: now 1790618378938874 us; error 38 us; skew 500 ppm
Sep 28 17:59:38 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:38.941643 21250 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:59:38.942092 21250 webserver.cc:460] Webserver started at http://127.5.46.65:44089/ using document root <none> and password file <none>
I20260928 17:59:38.942408 21250 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:38.942503 21250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:38.944027 21250 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:38.944809 21285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:38.945113 21250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:38.945240 21250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "900b5f7e24884850b933d54f7b5fcb72"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "5bc56aa4d55804cd2285d7b680870062"
server_key_iv: "63d2057c212b33f3bf77d3db0536c47d"
server_key_version: "encryptionkey@0"
I20260928 17:59:38.945611 21250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:38.963967 21250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:38.966624 21250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:38.966809 21250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:38.967075 21250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:38.967535 21292 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 17:59:38.969774 21250 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 17:59:38.969856 21250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 17:59:38.969905 21250 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 17:59:38.970556 21250 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 17:59:38.970614 21250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 17:59:38.970703 21292 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap starting.
I20260928 17:59:38.978181 21250 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33563
I20260928 17:59:38.978219 21399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33563 every 8 connection(s)
I20260928 17:59:38.978609 21250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:59:38.982654 21292 log.cc:824] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:38.985375 21292 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:38.985708 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21250
I20260928 17:59:38.985991 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:41179
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=39239
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:59:38 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618378, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:38.987407 21292 tablet_bootstrap.cc:492] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap complete.
I20260928 17:59:38.988682 21292 ts_tablet_manager.cc:1397] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.012s
I20260928 17:59:38.990262 21292 raft_consensus.cc:348] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:38.990633 21292 raft_consensus.cc:729] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900b5f7e24884850b933d54f7b5fcb72, State: Initialized, Role: FOLLOWER
I20260928 17:59:38.990810 21292 consensus_queue.cc:260] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } }
I20260928 17:59:38.991102 21292 ts_tablet_manager.cc:1428] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260928 17:59:38.991770 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:38.979415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:36285 (local address 127.5.46.126:46515)
0928 17:59:38.980324 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:38.980331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:38.980358 (+ 27us) server_negotiation.cc:407] Connection header received
0928 17:59:38.980468 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:38.980472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:38.980511 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:38.980652 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:38.981297 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.982192 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.983587 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.984337 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.984471 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:38.987444 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:38.987449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:38.987449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:38.987463 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:38.989698 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:38.990208 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:38.990212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:38.990212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:38.990328 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:38.990612 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:38.990613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:38.990613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:38.990910 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:38.990976 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:38.991445 (+ 469us) server_negotiation.cc:299] Negotiation successful
0928 17:59:38.991571 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":72,"threads_started":1}
I20260928 17:59:38.991768 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:38.979331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:36285)
0928 17:59:38.979594 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:38.979639 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:38.980158 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:38.980681 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:38.980685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:38.980982 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:38.981170 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:38.981175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.982347 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.982351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:38.983480 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:38.983483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.984468 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.984470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:38.984619 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:38.985308 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:38.985318 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:38.987308 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:38.989799 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:38.989804 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:38.989807 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:38.990094 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:38.990413 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:38.990414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:38.990414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:38.990533 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:38.991047 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:38.991052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:38.991213 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 17:59:38.991372 (+ 159us) client_negotiation.cc:240] Negotiation successful
0928 17:59:38.991557 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":87,"threads_started":1}
I20260928 17:59:38.992537 21400 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:38.992633 21400 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:38.992812 21400 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:38.993466 21212 ts_manager.cc:194] Registered new tserver with Master: 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563)
I20260928 17:59:38.994570 21212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:36285
I20260928 17:59:38.995394 21400 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
I20260928 17:59:38.999179 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:38.994260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.5.46.65:33563)
0928 17:59:38.996205 (+ 1945us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:38.996210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:38.996219 (+ 9us) server_negotiation.cc:407] Connection header received
0928 17:59:38.996251 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:38.996253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:38.996267 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:38.996546 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:38.996588 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.997252 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.998275 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.998750 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.998837 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:38.998893 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:38.999062 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 17:59:38.999099 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1919,"thread_start_us":68,"threads_started":1}
I20260928 17:59:38.999176 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:38.994042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33563 (local address 127.0.0.1:46664)
0928 17:59:38.995950 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:38.995992 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:38.996042 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:38.996372 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:38.996375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:38.996377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:38.996471 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:38.996473 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.997339 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.997340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:38.998162 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:38.998164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:38.998837 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:38.998838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:38.998932 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:38.998946 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 17:59:38.999044 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 17:59:38.999099 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1881,"thread_start_us":77,"threads_started":1}
I20260928 17:59:38.999820 21334 tablet_service.cc:1514] Processing DeleteTablet for tablet 54f2441bb7374fa58c0f5ecb1a557f1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:18 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46664
I20260928 17:59:39.000464 21409 tablet_replica.cc:330] stopping tablet replica
I20260928 17:59:39.000586 21409 raft_consensus.cc:2229] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:39.000654 21409 raft_consensus.cc:2258] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:39.001003 21409 ts_tablet_manager.cc:1905] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:39.003388 21409 ts_tablet_manager.cc:1918] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:39.003480 21409 log.cc:1196] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal/wals/54f2441bb7374fa58c0f5ecb1a557f1f
I20260928 17:59:39.003716 21409 ts_tablet_manager.cc:1939] T 54f2441bb7374fa58c0f5ecb1a557f1f P 900b5f7e24884850b933d54f7b5fcb72: Deleting consensus metadata
I20260928 17:59:39.004290 21198 catalog_manager.cc:4841] TS 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563): tablet 54f2441bb7374fa58c0f5ecb1a557f1f (table test-table [id=b7444720a21d41d49727dec7ff82a8d4]) successfully deleted
W20260928 17:59:39.086361 21405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:39.086581 21405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:39.086606 21405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:39.087806 21405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:39.087875 21405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:39.087904 21405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:59:39.089341 21405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:41179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=39239
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:39.089543 21405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:39.089737 21405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:39.094981 21417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:39.095055 21405 server_base.cc:1034] running on GCE node
W20260928 17:59:39.094988 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:39.094981 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:39.095431 21405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:39.095693 21405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:39.096839 21405 hybrid_clock.cc:648] HybridClock initialized: now 1790618379096826 us; error 41 us; skew 500 ppm
Sep 28 17:59:39 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:39.099530 21405 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:59:39.100078 21405 webserver.cc:460] Webserver started at http://127.5.46.66:39239/ using document root <none> and password file <none>
I20260928 17:59:39.100334 21405 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:39.100410 21405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:39.101895 21405 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:39.102635 21424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.102833 21405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:39.102938 21405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "2a61b8261dbf821a9b00388a582f9564"
server_key_iv: "3677d4f276c2b4a49e866e03b79d3524"
server_key_version: "encryptionkey@0"
I20260928 17:59:39.103312 21405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:39.121670 21405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:39.124330 21405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:39.124524 21405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:39.124804 21405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:39.125182 21405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:39.125223 21405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.125249 21405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:39.125304 21405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.131817 21405 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:41179
I20260928 17:59:39.131924 21537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:41179 every 8 connection(s)
I20260928 17:59:39.132344 21405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:59:39.133235 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21405
I20260928 17:59:39.133574 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:37417
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=39439
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 17:59:39 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:39.143875 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.133171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:42941 (local address 127.5.46.126:46515)
0928 17:59:39.133315 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.133318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.133510 (+ 192us) server_negotiation.cc:407] Connection header received
0928 17:59:39.134366 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.134367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.134381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.134428 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:39.135793 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.136394 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.137661 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.137913 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.138014 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.140535 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:39.140537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:39.140538 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:39.140547 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:39.142215 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.142722 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.142723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.142724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.142792 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.143081 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.143082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.143083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.143218 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:39.143298 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.143636 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.143738 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260928 17:59:39.144074 21540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.133135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:42941)
0928 17:59:39.133333 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.133393 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.134250 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.134509 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.134514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.135440 (+ 926us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:39.135667 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.135673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.136528 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.136529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.137551 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.137554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.138025 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.138027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.138173 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.138787 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:39.138796 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:39.140417 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:39.142346 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:39.142355 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:39.142358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:39.142629 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:39.142869 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:39.142870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:39.142871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:39.142997 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:39.143305 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:39.143313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:39.143470 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.143591 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.143772 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260928 17:59:39.144721 21538 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:39.144815 21538 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:39.145047 21538 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:39.145498 21212 ts_manager.cc:194] Registered new tserver with Master: 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
I20260928 17:59:39.146030 21212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:42941
W20260928 17:59:39.218263 21541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:39.218501 21541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:39.218529 21541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:39.220017 21541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:39.220072 21541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:39.220103 21541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 17:59:39.221763 21541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:37417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=39439
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:39.222069 21541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:39.222343 21541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:39.227972 21549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:39.228077 21541 server_base.cc:1034] running on GCE node
W20260928 17:59:39.227995 21547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:39.227972 21546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:39.228443 21541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:39.228650 21541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:39.229884 21541 hybrid_clock.cc:648] HybridClock initialized: now 1790618379229869 us; error 31 us; skew 500 ppm
Sep 28 17:59:39 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:39.232492 21541 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 17:59:39.233034 21541 webserver.cc:460] Webserver started at http://127.5.46.67:39439/ using document root <none> and password file <none>
I20260928 17:59:39.233325 21541 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:39.233409 21541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:39.234861 21541 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 17:59:39.235531 21556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.235756 21541 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:59:39.235877 21541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "f7bc3b1cadd240adb2119d7329736499"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "1064683d67b859bc4a4b1596c2971f90"
server_key_iv: "79850329c64fd34e437df6e92443f63e"
server_key_version: "encryptionkey@0"
I20260928 17:59:39.236233 21541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:39.257970 21541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:39.260566 21541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:39.260746 21541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:39.260994 21541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:39.261327 21541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:39.261381 21541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.261415 21541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:39.261442 21541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.267458 21541 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:37417
I20260928 17:59:39.267508 21669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:37417 every 8 connection(s)
I20260928 17:59:39.267915 21541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 17:59:39.269465 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21541
Sep 28 17:59:39 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618379, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 17:59:39.278184 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.269989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.5.46.126:46515)
0928 17:59:39.270131 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.270133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.270255 (+ 122us) server_negotiation.cc:407] Connection header received
0928 17:59:39.270320 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.270321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.270331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.270395 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:39.271114 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.272053 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.273200 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.273414 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.273514 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.275016 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:39.275017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:39.275017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:39.275024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:39.276930 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.277282 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.277283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.277284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.277334 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.277565 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.277567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.277567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.277729 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:39.277907 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.277946 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.278063 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260928 17:59:39.280018 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.268718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:36285 (local address 127.5.46.126:46515)
0928 17:59:39.268810 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.268813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.269097 (+ 284us) server_negotiation.cc:407] Connection header received
0928 17:59:39.269596 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.269598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.269611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.269665 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:39.271051 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.272053 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.273479 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.273708 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.273796 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.276529 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:39.276530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:39.276531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:39.276538 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:39.278117 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.278898 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.278899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.278899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.278947 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.279307 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.279308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.279308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.279405 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:39.279484 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.279815 (+ 331us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.279934 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":15,"server-negotiator.queue_time_us":52}
I20260928 17:59:39.280251 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.268759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:36285)
0928 17:59:39.268964 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.269000 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.269486 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.269755 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.269760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.270615 (+ 855us) client_negotiation.cc:189] Negotiated authn=SASL
0928 17:59:39.270920 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.270927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.272158 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.272159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.273401 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.273403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.273813 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.273814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.273918 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.274396 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 17:59:39.274408 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 17:59:39.276426 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 17:59:39.278223 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:39.278247 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:39.278249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:39.278674 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:39.279058 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 17:59:39.279059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 17:59:39.279059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 17:59:39.279238 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 17:59:39.279478 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 17:59:39.279484 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 17:59:39.279670 (+ 186us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.279783 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.279976 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":107,"threads_started":1}
I20260928 17:59:39.280820 21670 heartbeater.cc:347] Connected to a master server at 127.5.46.126:46515
I20260928 17:59:39.280930 21670 heartbeater.cc:464] Registering TS with master...
I20260928 17:59:39.281111 21670 heartbeater.cc:511] Master 127.5.46.126:46515 requested a full tablet report, sending...
I20260928 17:59:39.281522 21212 ts_manager.cc:194] Registered new tserver with Master: f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417)
I20260928 17:59:39.281893 21212 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:36285
I20260928 17:59:39.290088 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 17:59:39.292863 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.290497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46666 (local address 127.5.46.126:46515)
0928 17:59:39.290560 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.290562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.290671 (+ 109us) server_negotiation.cc:407] Connection header received
0928 17:59:39.290734 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.290735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.290772 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.290835 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:59:39.290931 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.291425 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.292268 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.292451 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.292505 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.292636 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:59:39.292680 (+ 44us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 17:59:39.292810 (+ 130us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46666: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260928 17:59:39.292939 21403 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46666: token expired
W20260928 17:59:39.293047 21141 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.46.126:46515)
I20260928 17:59:39.299249 21403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.293329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.5.46.126:46515)
0928 17:59:39.293365 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.293366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.293374 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:39.293393 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.293394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.293402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.293459 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 17:59:39.294011 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.294515 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.295269 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.295441 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.295497 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.296726 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 17:59:39.296727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 17:59:39.296728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 17:59:39.296734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 17:59:39.298147 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.298490 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.298490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.298491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.298541 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 17:59:39.298758 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 17:59:39.298759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 17:59:39.298759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 17:59:39.298897 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 17:59:39.298979 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.299124 (+ 145us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.299181 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 17:59:39.300626 21212 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 17:59:39.302168 21212 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 17:59:39.304551 21334 tablet_service.cc:1467] Processing CreateTablet for tablet efaa47ada9bb45269a100c5de819918f (DEFAULT_TABLE table=test-table [id=38942db392534e0fb8a9853ee4c3b404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:39.304783 21334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaa47ada9bb45269a100c5de819918f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:39.306238 21292 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap starting.
I20260928 17:59:39.306684 21292 tablet_bootstrap.cc:654] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:39.307301 21292 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: No bootstrap required, opened a new log
I20260928 17:59:39.307375 21292 ts_tablet_manager.cc:1397] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:39.307528 21292 raft_consensus.cc:348] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.307616 21292 raft_consensus.cc:374] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:39.307672 21292 raft_consensus.cc:729] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900b5f7e24884850b933d54f7b5fcb72, State: Initialized, Role: FOLLOWER
I20260928 17:59:39.307722 21292 consensus_queue.cc:260] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.307821 21292 ts_tablet_manager.cc:1428] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:39.308454 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.304179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.5.46.67:37417)
0928 17:59:39.304377 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.304382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.304506 (+ 124us) server_negotiation.cc:407] Connection header received
0928 17:59:39.304557 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.304560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.304577 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.304664 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:39.304914 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.305965 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.307171 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.307976 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.308052 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.308112 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.308297 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.308348 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260928 17:59:39.308514 21407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.304056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41179 (local address 127.0.0.1:57886)
0928 17:59:39.304203 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.304214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.304249 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.304614 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.304616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.304617 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:39.304723 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.304725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.306036 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.306037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.307260 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.307263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.308275 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.308276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.308336 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.308340 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.308397 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.308443 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":67}
I20260928 17:59:39.308531 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.304195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57886 (local address 127.5.46.66:41179)
0928 17:59:39.304470 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.304474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.304485 (+ 11us) server_negotiation.cc:407] Connection header received
0928 17:59:39.304527 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.304530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.304546 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.304622 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:39.304783 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.305932 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.307463 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.308198 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.308273 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.308328 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.308411 (+ 83us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.308447 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":103,"threads_started":1}
I20260928 17:59:39.309089 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.304056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37417 (local address 127.0.0.1:33930)
0928 17:59:39.304425 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.304435 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.304490 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.304797 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.304798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.304800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:39.304847 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.304849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.306057 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.306058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.307094 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.307098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.308090 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.308091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.308153 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.308157 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.308993 (+ 836us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.309036 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":23,"thread_start_us":137,"threads_started":1}
I20260928 17:59:39.309341 21472 tablet_service.cc:1467] Processing CreateTablet for tablet efaa47ada9bb45269a100c5de819918f (DEFAULT_TABLE table=test-table [id=38942db392534e0fb8a9853ee4c3b404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:39.309707 21472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaa47ada9bb45269a100c5de819918f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:39.309844 21604 tablet_service.cc:1467] Processing CreateTablet for tablet efaa47ada9bb45269a100c5de819918f (DEFAULT_TABLE table=test-table [id=38942db392534e0fb8a9853ee4c3b404]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 17:59:39.310176 21604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet efaa47ada9bb45269a100c5de819918f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 17:59:39.312793 21680 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Bootstrap starting.
I20260928 17:59:39.312960 21681 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Bootstrap starting.
I20260928 17:59:39.313436 21680 tablet_bootstrap.cc:654] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:39.313570 21681 tablet_bootstrap.cc:654] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Neither blocks nor log segments found. Creating new log.
I20260928 17:59:39.314007 21680 log.cc:824] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:39.314133 21681 log.cc:824] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:39.314910 21680 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: No bootstrap required, opened a new log
I20260928 17:59:39.314920 21681 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: No bootstrap required, opened a new log
I20260928 17:59:39.314965 21681 ts_tablet_manager.cc:1397] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:59:39.314997 21680 ts_tablet_manager.cc:1397] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 17:59:39.316655 21681 raft_consensus.cc:348] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.316811 21681 raft_consensus.cc:374] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:39.316835 21681 raft_consensus.cc:729] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7bc3b1cadd240adb2119d7329736499, State: Initialized, Role: FOLLOWER
I20260928 17:59:39.316720 21680 raft_consensus.cc:348] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.316896 21680 raft_consensus.cc:374] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 17:59:39.316941 21680 raft_consensus.cc:729] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86fa9353a3f7497ab8ef9ea317a3543d, State: Initialized, Role: FOLLOWER
I20260928 17:59:39.316967 21681 consensus_queue.cc:260] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.317054 21680 consensus_queue.cc:260] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.317282 21681 ts_tablet_manager.cc:1428] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:59:39.317348 21680 ts_tablet_manager.cc:1428] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 17:59:39.317363 21538 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
I20260928 17:59:39.317425 21670 heartbeater.cc:503] Master 127.5.46.126:46515 was elected leader, sending a full tablet report...
W20260928 17:59:39.382988 21539 tablet.cc:2369] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:59:39.470808 21406 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 17:59:39.470950 21406 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.471374 21406 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
I20260928 17:59:39.475143 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.471594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37417 (local address 127.5.46.65:42019)
0928 17:59:39.471659 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.471672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.471736 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.471955 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.471957 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.471959 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:39.472046 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.472048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.472860 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.472860 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.474013 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.474015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.474913 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.474914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.474969 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.474973 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.475016 (+ 43us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.475047 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":4}
I20260928 17:59:39.475206 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.471714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:42019 (local address 127.5.46.67:37417)
0928 17:59:39.471771 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.471773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.471781 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:39.471815 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.471816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.471828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.471928 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:39.472133 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.472767 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.474141 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.474818 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.474910 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.474923 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.475057 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.475118 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 17:59:39.475466 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.471715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:52093 (local address 127.5.46.66:41179)
0928 17:59:39.471763 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.471766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.471803 (+ 37us) server_negotiation.cc:407] Connection header received
0928 17:59:39.471892 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.471893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.471905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.471959 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 17:59:39.472452 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.473061 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.474327 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.475063 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.475146 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.475157 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.475362 (+ 205us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.475406 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 17:59:39.475571 21624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaa47ada9bb45269a100c5de819918f" candidate_uuid: "900b5f7e24884850b933d54f7b5fcb72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bc3b1cadd240adb2119d7329736499" is_pre_election: true
I20260928 17:59:39.475697 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.471594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41179 (local address 127.5.46.65:52093)
0928 17:59:39.471748 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.471756 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:39.471803 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:39.471973 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 17:59:39.471974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 17:59:39.471975 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 17:59:39.472363 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.472365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.473142 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.473143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.474224 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 17:59:39.474226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.475165 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.475166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 17:59:39.475255 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.475260 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 17:59:39.475546 (+ 286us) client_negotiation.cc:240] Negotiation successful
0928 17:59:39.475586 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":23,"thread_start_us":82,"threads_started":1}
I20260928 17:59:39.475756 21624 raft_consensus.cc:2454] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900b5f7e24884850b933d54f7b5fcb72 in term 0.
I20260928 17:59:39.476045 21286 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72, f7bc3b1cadd240adb2119d7329736499; no voters:
I20260928 17:59:39.476071 21492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaa47ada9bb45269a100c5de819918f" candidate_uuid: "900b5f7e24884850b933d54f7b5fcb72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" is_pre_election: true
I20260928 17:59:39.476217 21492 raft_consensus.cc:2454] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 900b5f7e24884850b933d54f7b5fcb72 in term 0.
I20260928 17:59:39.476233 21406 raft_consensus.cc:2781] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 17:59:39.476325 21406 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 17:59:39.476361 21406 raft_consensus.cc:3037] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:39.477133 21406 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.477293 21406 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 1 election: Requested vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
I20260928 17:59:39.477437 21492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaa47ada9bb45269a100c5de819918f" candidate_uuid: "900b5f7e24884850b933d54f7b5fcb72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
I20260928 17:59:39.477501 21624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "efaa47ada9bb45269a100c5de819918f" candidate_uuid: "900b5f7e24884850b933d54f7b5fcb72" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7bc3b1cadd240adb2119d7329736499"
I20260928 17:59:39.477530 21492 raft_consensus.cc:3037] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:39.477572 21624 raft_consensus.cc:3037] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 0 FOLLOWER]: Advancing to term 1
I20260928 17:59:39.478457 21624 raft_consensus.cc:2454] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900b5f7e24884850b933d54f7b5fcb72 in term 1.
I20260928 17:59:39.478457 21492 raft_consensus.cc:2454] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 900b5f7e24884850b933d54f7b5fcb72 in term 1.
I20260928 17:59:39.478680 21288 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86fa9353a3f7497ab8ef9ea317a3543d, 900b5f7e24884850b933d54f7b5fcb72; no voters:
I20260928 17:59:39.478843 21406 raft_consensus.cc:2781] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 17:59:39.479033 21406 raft_consensus.cc:686] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 LEADER]: Becoming Leader. State: Replica: 900b5f7e24884850b933d54f7b5fcb72, State: Running, Role: LEADER
I20260928 17:59:39.479097 21406 consensus_queue.cc:237] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:39.479992 21212 catalog_manager.cc:5495] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 reported cstate change: term changed from 0 to 1, leader changed from <none> to 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65). New cstate: current_term: 1 leader_uuid: "900b5f7e24884850b933d54f7b5fcb72" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } health_report { overall_health: HEALTHY } } }
W20260928 17:59:39.483687 21401 tablet.cc:2369] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 17:59:39.493170 21408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.490780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.5.46.65:33563)
0928 17:59:39.490845 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 17:59:39.490847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 17:59:39.490855 (+ 8us) server_negotiation.cc:407] Connection header received
0928 17:59:39.490891 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 17:59:39.490892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 17:59:39.490912 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 17:59:39.490973 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 17:59:39.491092 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.491560 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.492422 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 17:59:39.492607 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 17:59:39.492661 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 17:59:39.492833 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 17:59:39.492931 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 17:59:39.492987 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 17:59:39.493090 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 17:59:39.493110 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 17:59:39.494837 21492 raft_consensus.cc:1261] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Refusing update from remote peer 900b5f7e24884850b933d54f7b5fcb72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:39.494889 21624 raft_consensus.cc:1261] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Refusing update from remote peer 900b5f7e24884850b933d54f7b5fcb72: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 17:59:39.495247 21406 consensus_queue.cc:1035] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:39.495414 21687 consensus_queue.cc:1035] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 17:59:39.498595 21692 mvcc.cc:205] Tried to move back new op lower bound from 7334372882408050688 to 7334372882347450368. Current Snapshot: MvccSnapshot[applied={T|T < 7334372882408050688 or (T in {7334372882408050688})}]
I20260928 17:59:39.500842 21212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:39.500960 21212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 17:59:39.501708 21212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Sending DeleteTablet for 3 replicas of tablet efaa47ada9bb45269a100c5de819918f
I20260928 17:59:39.502033 21604 tablet_service.cc:1514] Processing DeleteTablet for tablet efaa47ada9bb45269a100c5de819918f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:39 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:33930
I20260928 17:59:39.502089 21334 tablet_service.cc:1514] Processing DeleteTablet for tablet efaa47ada9bb45269a100c5de819918f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:39 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:46664
I20260928 17:59:39.502086 21472 tablet_service.cc:1514] Processing DeleteTablet for tablet efaa47ada9bb45269a100c5de819918f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 17:59:39 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:57886
I20260928 17:59:39.502219 21409 tablet_replica.cc:330] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: stopping tablet replica
I20260928 17:59:39.502275 21409 raft_consensus.cc:2229] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 LEADER]: Raft consensus shutting down.
I20260928 17:59:39.502326 21700 tablet_replica.cc:330] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: stopping tablet replica
I20260928 17:59:39.502381 21409 raft_consensus.cc:2258] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:39.502413 21700 raft_consensus.cc:2229] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:39.502568 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21250
I20260928 17:59:39.502614 21700 raft_consensus.cc:2258] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:39.502671 21409 ts_tablet_manager.cc:1905] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:39.503674 21699 tablet_replica.cc:330] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: stopping tablet replica
I20260928 17:59:39.503800 21699 raft_consensus.cc:2229] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 17:59:39.503955 21699 raft_consensus.cc:2258] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 17:59:39.504513 21699 ts_tablet_manager.cc:1905] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:39.505160 21700 ts_tablet_manager.cc:1905] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 17:59:39.506569 21699 ts_tablet_manager.cc:1918] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:39.506692 21699 log.cc:1196] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/efaa47ada9bb45269a100c5de819918f
I20260928 17:59:39.506961 21699 ts_tablet_manager.cc:1939] T efaa47ada9bb45269a100c5de819918f P f7bc3b1cadd240adb2119d7329736499: Deleting consensus metadata
I20260928 17:59:39.507167 21700 ts_tablet_manager.cc:1918] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 17:59:39.507249 21700 log.cc:1196] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/efaa47ada9bb45269a100c5de819918f
I20260928 17:59:39.507457 21196 catalog_manager.cc:4841] TS f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): tablet efaa47ada9bb45269a100c5de819918f (table test-table [id=38942db392534e0fb8a9853ee4c3b404]) successfully deleted
I20260928 17:59:39.507510 21700 ts_tablet_manager.cc:1939] T efaa47ada9bb45269a100c5de819918f P 86fa9353a3f7497ab8ef9ea317a3543d: Deleting consensus metadata
I20260928 17:59:39.507900 21198 catalog_manager.cc:4841] TS 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): tablet efaa47ada9bb45269a100c5de819918f (table test-table [id=38942db392534e0fb8a9853ee4c3b404]) successfully deleted
W20260928 17:59:39.508466 21198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:33563 (error 108)
I20260928 17:59:39.508718 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21405
I20260928 17:59:39.508988 21676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:39.508868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:33563 (local address 127.0.0.1:46688)
0928 17:59:39.508920 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:39.508951 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 17:59:39.509090 21198 catalog_manager.cc:4584] TS 900b5f7e24884850b933d54f7b5fcb72 (127.5.46.65:33563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet efaa47ada9bb45269a100c5de819918f: Network error: Client connection negotiation failed: client connection to 127.5.46.65:33563: connect: Connection refused (error 111)
I20260928 17:59:39.514096 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21541
I20260928 17:59:39.518985 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21179
I20260928 17:59:59.524698 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--webserver_interface=127.5.46.126
--webserver_port=43739
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:59.605793 21702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:59.606045 21702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:59.606081 21702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:59.607247 21702 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 17:59:59.607321 21702 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 17:59:59.607343 21702 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 17:59:59.607364 21702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 17:59:59.607396 21702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:59.607427 21702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:59.607455 21702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 17:59:59.607481 21702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 17:59:59.609110 21702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=43739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:59.609386 21702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:59.609628 21702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:59.614888 21710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:59.615062 21702 server_base.cc:1034] running on GCE node
W20260928 17:59:59.614936 21708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:59.614892 21707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:59.615402 21702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:59.615703 21702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:59.616901 21702 hybrid_clock.cc:648] HybridClock initialized: now 1790618399616863 us; error 57 us; skew 500 ppm
Sep 28 17:59:59 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:59.620018 21702 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 17:59:59.620448 21702 webserver.cc:460] Webserver started at http://127.5.46.126:43739/ using document root <none> and password file <none>
I20260928 17:59:59.620677 21702 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:59.620725 21702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:59.622180 21702 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:59.622965 21717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:59.623247 21702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:59.623390 21702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "a743e415acd44224a1bffec4848d035e"
format_stamp: "Formatted at 2026-09-28 17:59:17 on dist-test-slave-z2t3"
server_key: "e6d6765e5032036ce2dc55b255ad0e8f"
server_key_iv: "6b41590903598e92480f9e4a526a16c0"
server_key_version: "encryptionkey@0"
I20260928 17:59:59.623816 21702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:59.640331 21702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:59.642881 21702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:59.643080 21702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:59.647460 21702 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46515
I20260928 17:59:59.647497 21769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46515 every 8 connection(s)
I20260928 17:59:59.647960 21702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 17:59:59.649483 21770 sys_catalog.cc:263] Verifying existing consensus state
I20260928 17:59:59.650041 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21702
I20260928 17:59:59.650213 21770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap starting.
I20260928 17:59:59.650403 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:33563
--local_ip_for_outbound_sockets=127.5.46.65
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=44089
--webserver_interface=127.5.46.65
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 17:59:59.659231 21770 log.cc:824] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:59.661592 21770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:59.661934 21770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Bootstrap complete.
I20260928 17:59:59.664320 21770 raft_consensus.cc:348] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:59.664551 21770 raft_consensus.cc:729] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a743e415acd44224a1bffec4848d035e, State: Initialized, Role: FOLLOWER
I20260928 17:59:59.664723 21770 consensus_queue.cc:260] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:59.664826 21770 raft_consensus.cc:388] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 17:59:59.664878 21770 raft_consensus.cc:482] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 17:59:59.664928 21770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 2 FOLLOWER]: Advancing to term 3
I20260928 17:59:59.665999 21770 raft_consensus.cc:504] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:59.666092 21770 leader_election.cc:302] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a743e415acd44224a1bffec4848d035e; no voters:
I20260928 17:59:59.666345 21770 leader_election.cc:288] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 17:59:59.666386 21774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 3 FOLLOWER]: Leader election won for term 3
I20260928 17:59:59.666528 21774 raft_consensus.cc:686] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [term 3 LEADER]: Becoming Leader. State: Replica: a743e415acd44224a1bffec4848d035e, State: Running, Role: LEADER
I20260928 17:59:59.666641 21774 consensus_queue.cc:237] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } }
I20260928 17:59:59.666706 21770 sys_catalog.cc:564] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 17:59:59.667059 21775 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:59.667073 21774 sys_catalog.cc:455] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a743e415acd44224a1bffec4848d035e. Latest consensus state: current_term: 3 leader_uuid: "a743e415acd44224a1bffec4848d035e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a743e415acd44224a1bffec4848d035e" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46515 } } }
I20260928 17:59:59.667312 21774 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:59.667294 21775 sys_catalog.cc:458] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e [sys.catalog]: This master's current role is: LEADER
I20260928 17:59:59.667670 21783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 17:59:59.668606 21783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 17:59:59.668771 21783 catalog_manager.cc:1259] Loaded cluster ID: 36e098c1f7a649009327b3410085126c
I20260928 17:59:59.668972 21783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 17:59:59.670349 21783 catalog_manager.cc:1504] Loading token signing keys...
I20260928 17:59:59.670584 21783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Loaded TSK: 0
I20260928 17:59:59.670651 21783 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a743e415acd44224a1bffec4848d035e: Loaded TSK: 1
F20260928 17:59:59.670921 21783 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f57a89609ad google::LogMessage::Fail() at ??:0
@ 0x7f57a896388a google::LogMessage::SendToLog() at ??:0
@ 0x7f57a89604de google::LogMessage::Flush() at ??:0
@ 0x7f57a8961a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f57ab0cd104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f57ab0cad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f57b08be1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f57b08be48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f57b08be594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f57b0884c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f57b088ead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f57aa82e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f57aa823954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f57ab6876db start_thread at ??:0
@ 0x7f57a745e61f clone at ??:0
W20260928 17:59:59.754370 21772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:59.754614 21772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:59.754653 21772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:59.755950 21772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:59.756013 21772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:59.756060 21772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 17:59:59.757643 21772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:33563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=44089
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:59.757926 21772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:59.758176 21772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:59.764375 21811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:59.764469 21812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:59.764565 21772 server_base.cc:1034] running on GCE node
W20260928 17:59:59.764766 21814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:59.765002 21772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:59.765281 21772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:59.766463 21772 hybrid_clock.cc:648] HybridClock initialized: now 1790618399766439 us; error 42 us; skew 500 ppm
Sep 28 17:59:59 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:59.769515 21772 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 17:59:59.770077 21772 webserver.cc:460] Webserver started at http://127.5.46.65:44089/ using document root <none> and password file <none>
I20260928 17:59:59.770310 21772 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:59.770380 21772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:59.771795 21772 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:59.772637 21821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:59.772809 21772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:59.772910 21772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "900b5f7e24884850b933d54f7b5fcb72"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "5bc56aa4d55804cd2285d7b680870062"
server_key_iv: "63d2057c212b33f3bf77d3db0536c47d"
server_key_version: "encryptionkey@0"
I20260928 17:59:59.773277 21772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:59.782543 21772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:59.792375 21772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:59.792562 21772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:59.792826 21772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:59.793303 21828 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 17:59:59.795284 21772 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 17:59:59.795356 21772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 17:59:59.795404 21772 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 17:59:59.796188 21772 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 17:59:59.796243 21772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260928 17:59:59.796327 21828 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap starting.
I20260928 17:59:59.798267 21828 log.cc:824] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Log is configured to *not* fsync() on all Append() calls
I20260928 17:59:59.801326 21828 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 17:59:59.801643 21828 tablet_bootstrap.cc:492] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Bootstrap complete.
I20260928 17:59:59.801815 21828 ts_tablet_manager.cc:1397] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260928 17:59:59.803520 21772 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:33563
I20260928 17:59:59.803551 21936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:33563 every 8 connection(s)
I20260928 17:59:59.804008 21772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 17:59:59.803674 21828 raft_consensus.cc:348] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:59.804193 21828 raft_consensus.cc:729] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900b5f7e24884850b933d54f7b5fcb72, State: Initialized, Role: FOLLOWER
I20260928 17:59:59.804507 21828 consensus_queue.cc:260] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 17:59:59.804839 21828 ts_tablet_manager.cc:1428] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
I20260928 17:59:59.807000 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21772
I20260928 17:59:59.807296 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:41179
--local_ip_for_outbound_sockets=127.5.46.66
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=39239
--webserver_interface=127.5.46.66
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 17:59:59.872325 21940 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 17:59:59.804908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:47133)
0928 17:59:59.805059 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.805102 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 17:59:59.805241 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 17:59:59.872106 (+ 66865us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
W20260928 17:59:59.872627 21937 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46515 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:46515: Client connection negotiation failed: client connection to 127.5.46.126:46515: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 17:59:59.872957 21940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:59.872836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:55473)
0928 17:59:59.872892 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.872928 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 17:59:59.873327 21940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:59.873240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:50273)
0928 17:59:59.873289 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.873311 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 17:59:59.873457 21937 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 17:59:59.893005 21941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 17:59:59.893253 21941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 17:59:59.893294 21941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 17:59:59.894481 21941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 17:59:59.894546 21941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 17:59:59.894593 21941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 17:59:59.896154 21941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:41179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=39239
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 17:59:59.896423 21941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 17:59:59.896667 21941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 17:59:59.901945 21946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:59.901954 21947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 17:59:59.901958 21949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 17:59:59.902350 21941 server_base.cc:1034] running on GCE node
I20260928 17:59:59.902578 21941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 17:59:59.902828 21941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 17:59:59.903990 21941 hybrid_clock.cc:648] HybridClock initialized: now 1790618399903977 us; error 28 us; skew 500 ppm
Sep 28 17:59:59 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 17:59:59.906455 21941 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 17:59:59.906898 21941 webserver.cc:460] Webserver started at http://127.5.46.66:39239/ using document root <none> and password file <none>
I20260928 17:59:59.907138 21941 fs_manager.cc:359] Metadata directory not provided
I20260928 17:59:59.907217 21941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 17:59:59.908628 21941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260928 17:59:59.909271 21956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:59.909456 21941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 17:59:59.909559 21941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "86fa9353a3f7497ab8ef9ea317a3543d"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "2a61b8261dbf821a9b00388a582f9564"
server_key_iv: "3677d4f276c2b4a49e866e03b79d3524"
server_key_version: "encryptionkey@0"
I20260928 17:59:59.909905 21941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 17:59:59.933887 21941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 17:59:59.935989 21941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 17:59:59.936165 21941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 17:59:59.936425 21941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 17:59:59.936750 21941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 17:59:59.936812 21941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:59.936856 21941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 17:59:59.936888 21941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 17:59:59.943051 21941 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:41179
I20260928 17:59:59.943166 22069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:41179 every 8 connection(s)
I20260928 17:59:59.943507 21941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 17:59:59.944639 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:59.944251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:51363)
0928 17:59:59.944420 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.944492 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
W20260928 17:59:59.945078 22070 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46515 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:46515: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
I20260928 17:59:59.945351 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:59.945258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:53247)
0928 17:59:59.945309 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.945334 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 17:59:59.945823 22072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 17:59:59.945706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:59939)
0928 17:59:59.945772 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 17:59:59.945802 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260928 17:59:59.946074 22070 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 17:59:59.953212 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 21941
I20260928 17:59:59.953447 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:37417
--local_ip_for_outbound_sockets=127.5.46.67
--tserver_master_addrs=127.5.46.126:46515
--webserver_port=39439
--webserver_interface=127.5.46.67
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:00.039402 22073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:00.039664 22073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:00.039696 22073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:00.041110 22073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:00.041185 22073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:00.041220 22073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:00.042907 22073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:37417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=39439
--tserver_master_addrs=127.5.46.126:46515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:00.043161 22073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:00.043375 22073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:00.049187 22079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:00.049253 22081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:00.049206 22078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:00.049322 22073 server_base.cc:1034] running on GCE node
I20260928 18:00:00.049542 22073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:00.049852 22073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:00.051014 22073 hybrid_clock.cc:648] HybridClock initialized: now 1790618400050986 us; error 41 us; skew 500 ppm
Sep 28 18:00:00 dist-test-slave-z2t3 krb5kdc[20631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618400, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:00.054124 22073 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 18:00:00.054651 22073 webserver.cc:460] Webserver started at http://127.5.46.67:39439/ using document root <none> and password file <none>
I20260928 18:00:00.054925 22073 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:00.055009 22073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:00.056595 22073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:00.057381 22088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:00.057613 22073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:00.057722 22073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "f7bc3b1cadd240adb2119d7329736499"
format_stamp: "Formatted at 2026-09-28 17:59:18 on dist-test-slave-z2t3"
server_key: "1064683d67b859bc4a4b1596c2971f90"
server_key_iv: "79850329c64fd34e437df6e92443f63e"
server_key_version: "encryptionkey@0"
I20260928 18:00:00.058166 22073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:00.076988 22073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:00.079351 22073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:00.079533 22073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:00.079962 22073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:00.080339 22073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:00.080405 22073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:00.080453 22073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:00.080479 22073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:00.086436 22073 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:37417
I20260928 18:00:00.086488 22201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:37417 every 8 connection(s)
I20260928 18:00:00.086869 22073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:00.088045 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:00.087589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:58253)
0928 18:00:00.087780 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:00.087892 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":69,"threads_started":1}
W20260928 18:00:00.088541 22202 heartbeater.cc:644] Failed to heartbeat to 127.5.46.126:46515 (0 consecutive failures): Network error: Failed to ping master at 127.5.46.126:46515: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
I20260928 18:00:00.088969 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:00.088838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:45861)
0928 18:00:00.088899 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:00.088937 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 18:00:00.089008 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22073
I20260928 18:00:00.089404 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:00.089301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:44877)
0928 18:00:00.089363 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:00.089387 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 18:00:00.089519 22202 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 18:00:00.874197 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:00.873909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:33465)
0928 18:00:00.874079 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:00.874130 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260928 18:00:00.946730 22207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:00.946451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:50457)
0928 18:00:00.946614 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:00.946664 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260928 18:00:01.090173 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.089921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:52887)
0928 18:00:01.090067 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:01.090105 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
I20260928 18:00:01.143723 22209 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:01.143919 22209 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 18:00:01.144308 22209 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
Sep 28 18:00:01 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.67@KRBTEST.COM
Sep 28 18:00:01 dist-test-slave-z2t3 krb5kdc[20631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618399, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.66@KRBTEST.COM
I20260928 18:00:01.155444 22212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.144708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:32845 (local address 127.5.46.67:37417)
0928 18:00:01.144842 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:01.144849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:01.144876 (+ 27us) server_negotiation.cc:407] Connection header received
0928 18:00:01.145408 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:01.145411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:01.145473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:01.145626 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:01.146654 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.147491 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.148515 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.148930 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.149065 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:01.151544 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:01.151549 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:01.151549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:01.151562 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:01.153704 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154173 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:01.154177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:01.154177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:01.154279 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154532 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:01.154533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:01.154534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:01.154793 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:01.154881 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:01.155178 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 18:00:01.155318 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":59,"threads_started":1}
I20260928 18:00:01.155444 22210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.144522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37417 (local address 127.5.46.65:32845)
0928 18:00:01.144676 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:01.144743 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:01.144805 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:01.145755 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:01.145756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:01.146116 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:01.146505 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:01.146507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.147593 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.147595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:01.148419 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:01.148421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.149042 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.149042 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:01.149182 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:01.149796 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:01.149806 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:01.151422 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:01.153803 (+ 2381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:01.153812 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:01.153814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:01.154087 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:01.154350 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:01.154351 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:01.154467 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:01.154873 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:01.154880 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:01.155012 (+ 132us) client_negotiation.cc:769] Sending connection context
0928 18:00:01.155119 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 18:00:01.155332 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":58,"thread_start_us":78,"threads_started":1}
I20260928 18:00:01.155483 22211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.144628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:47565 (local address 127.5.46.66:41179)
0928 18:00:01.144759 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:01.144772 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:01.144810 (+ 38us) server_negotiation.cc:407] Connection header received
0928 18:00:01.145408 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:01.145411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:01.145473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:01.145633 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:01.146519 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.147400 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.148516 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.148959 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.149112 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:01.151913 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:01.151919 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:01.151919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:01.151934 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:01.153796 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154211 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:01.154215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:01.154215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:01.154327 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154544 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:01.154545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:01.154545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:01.154793 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:01.154887 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:01.155189 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 18:00:01.155348 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260928 18:00:01.155561 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.144499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:41179 (local address 127.5.46.65:47565)
0928 18:00:01.144555 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:01.144567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:01.144637 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:01.145681 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:01.145684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:01.146113 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:01.146351 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:01.146357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.147524 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.147525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:01.148419 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:01.148421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:01.149073 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:01.149074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:01.149178 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:01.149860 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:01.149865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:01.151769 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:01.153886 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:01.153887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:01.153887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:01.154127 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:01.154407 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:01.154407 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:01.154408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:01.154474 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:01.154867 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:01.154875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:01.155012 (+ 137us) client_negotiation.cc:769] Sending connection context
0928 18:00:01.155120 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 18:00:01.155415 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":165}
W20260928 18:00:01.156424 21824 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
W20260928 18:00:01.156503 21822 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
I20260928 18:00:01.156531 21824 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72; no voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499
I20260928 18:00:01.156642 22209 raft_consensus.cc:2726] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 18:00:01.875075 22213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.874794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:57213)
0928 18:00:01.874960 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:01.875006 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260928 18:00:01.947585 22214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:01.947319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:38525)
0928 18:00:01.947489 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:01.947537 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":74,"threads_started":1}
I20260928 18:00:02.091061 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:02.090805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:48671)
0928 18:00:02.090965 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:02.091013 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260928 18:00:02.875975 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:02.875632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:42257)
0928 18:00:02.875822 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:02.875922 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
I20260928 18:00:02.948377 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:02.948142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:50047)
0928 18:00:02.948279 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:02.948329 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260928 18:00:03.091840 22218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:03.091582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:60845)
0928 18:00:03.091720 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:03.091759 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
I20260928 18:00:03.245931 22219 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:03.246083 22219 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 18:00:03.246309 22219 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
W20260928 18:00:03.246735 21824 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
W20260928 18:00:03.246824 21822 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
I20260928 18:00:03.246924 21822 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72; no voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499
I20260928 18:00:03.247030 22219 raft_consensus.cc:2726] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 18:00:03.876816 22220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:03.876543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:48217)
0928 18:00:03.876708 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:03.876764 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260928 18:00:03.949182 22221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:03.948924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:46965)
0928 18:00:03.949086 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:03.949134 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":66,"threads_started":1}
I20260928 18:00:04.092705 22222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:04.092445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:44711)
0928 18:00:04.092613 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:04.092651 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260928 18:00:04.844246 22223 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:04.844352 22223 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 18:00:04.844592 22223 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
W20260928 18:00:04.844985 21822 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
W20260928 18:00:04.845081 21824 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
I20260928 18:00:04.845165 21824 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72; no voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499
I20260928 18:00:04.845244 22223 raft_consensus.cc:2726] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 18:00:04.877641 22224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:04.877419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:57849)
0928 18:00:04.877548 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:04.877593 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260928 18:00:04.949986 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:04.949729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:56861)
0928 18:00:04.949895 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:04.949937 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260928 18:00:05.093559 22226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:05.093273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:37353)
0928 18:00:05.093461 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:05.093511 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260928 18:00:05.878444 22227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:05.878187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:48887)
0928 18:00:05.878332 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:05.878392 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260928 18:00:05.950733 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:05.950492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:56201)
0928 18:00:05.950636 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:05.950683 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260928 18:00:06.094506 22229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:06.094242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:45067)
0928 18:00:06.094409 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:06.094457 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":69,"threads_started":1}
I20260928 18:00:06.879297 22230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:06.879076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:44305)
0928 18:00:06.879209 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:06.879252 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260928 18:00:06.951532 22231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:06.951310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:43743)
0928 18:00:06.951445 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:06.951486 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260928 18:00:07.095399 22232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:07.095151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:40859)
0928 18:00:07.095296 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:07.095346 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260928 18:00:07.226878 22233 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:07.226986 22233 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 18:00:07.227219 22233 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
W20260928 18:00:07.227620 21822 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
W20260928 18:00:07.227710 21824 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
I20260928 18:00:07.227794 21824 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72; no voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499
I20260928 18:00:07.227916 22233 raft_consensus.cc:2726] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 18:00:07.880200 22234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:07.879967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:37063)
0928 18:00:07.880100 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:07.880148 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260928 18:00:07.952454 22235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:07.952155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:46915)
0928 18:00:07.952354 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:07.952406 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260928 18:00:08.096423 22236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:08.096153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:36413)
0928 18:00:08.096330 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:08.096374 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260928 18:00:08.881007 22237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:08.880743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:51673)
0928 18:00:08.880916 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:08.880961 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260928 18:00:08.953408 22238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:08.953152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:51187)
0928 18:00:08.953312 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:08.953358 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260928 18:00:09.097275 22239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:09.097018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:56825)
0928 18:00:09.097162 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:09.097207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260928 18:00:09.881929 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:09.881655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:40399)
0928 18:00:09.881836 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:09.881883 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260928 18:00:09.954178 22241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:09.953968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:51191)
0928 18:00:09.954090 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:09.954131 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260928 18:00:10.098134 22242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:10.097861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:40375)
0928 18:00:10.098031 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:10.098082 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260928 18:00:10.882805 22243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:10.882532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:59385)
0928 18:00:10.882709 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:10.882757 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":73,"threads_started":1}
I20260928 18:00:10.955091 22244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:10.954812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:35573)
0928 18:00:10.954997 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:10.955042 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260928 18:00:11.099206 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:11.098924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:51885)
0928 18:00:11.099105 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:11.099156 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
I20260928 18:00:11.883687 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:11.883435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:51221)
0928 18:00:11.883593 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:11.883639 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260928 18:00:11.956089 22247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:11.955760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:54957)
0928 18:00:11.955984 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:11.956040 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
I20260928 18:00:12.100183 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:12.099916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:53353)
0928 18:00:12.100082 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:12.100131 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260928 18:00:12.884526 22249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:12.884262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:49219)
0928 18:00:12.884430 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:12.884476 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":67,"threads_started":1}
I20260928 18:00:12.956910 22250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:12.956642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:43363)
0928 18:00:12.956807 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:12.956851 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260928 18:00:13.101023 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:13.100730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:60275)
0928 18:00:13.100926 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:13.100973 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
I20260928 18:00:13.124481 22252 raft_consensus.cc:482] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:13.124580 22252 raft_consensus.cc:504] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7bc3b1cadd240adb2119d7329736499" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37417 } } peers { permanent_uuid: "86fa9353a3f7497ab8ef9ea317a3543d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 41179 } } peers { permanent_uuid: "900b5f7e24884850b933d54f7b5fcb72" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 33563 } }
I20260928 18:00:13.124778 22252 leader_election.cc:288] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417), 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179)
W20260928 18:00:13.125268 21822 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f7bc3b1cadd240adb2119d7329736499 (127.5.46.67:37417): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
W20260928 18:00:13.125358 21824 leader_election.cc:341] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86fa9353a3f7497ab8ef9ea317a3543d (127.5.46.66:41179): Not found: Tablet not found: efaa47ada9bb45269a100c5de819918f
I20260928 18:00:13.125392 21824 leader_election.cc:302] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 900b5f7e24884850b933d54f7b5fcb72; no voters: 86fa9353a3f7497ab8ef9ea317a3543d, f7bc3b1cadd240adb2119d7329736499
I20260928 18:00:13.125509 22252 raft_consensus.cc:2726] T efaa47ada9bb45269a100c5de819918f P 900b5f7e24884850b933d54f7b5fcb72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 18:00:13.885382 22253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:13.885128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:49615)
0928 18:00:13.885291 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:13.885337 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260928 18:00:13.957818 22254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:13.957554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:45951)
0928 18:00:13.957723 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:13.957770 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":91,"threads_started":1}
I20260928 18:00:14.102032 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:14.101737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:46423)
0928 18:00:14.101926 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:14.101978 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
I20260928 18:00:14.886114 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:14.885870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.65:37219)
0928 18:00:14.886017 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:14.886057 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260928 18:00:14.958704 22257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:14.958459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.66:39639)
0928 18:00:14.958620 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:14.958657 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260928 18:00:15.097918 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21772
I20260928 18:00:15.102988 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:15.102722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46515 (local address 127.5.46.67:45439)
0928 18:00:15.102887 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:15.102928 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.126:46515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260928 18:00:15.103247 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 21941
I20260928 18:00:15.108623 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22073
2026-09-28T18:00:15Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58476 ms)
[----------] 30 tests from SecurityITest (238024 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:15 dist-test-slave-z2t3 krb5kdc[22261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:15 dist-test-slave-z2t3 krb5kdc[22261](info): set up 2 sockets
Sep 28 18:00:15 dist-test-slave-z2t3 krb5kdc[22261](info): commencing operation
krb5kdc: starting...
W20260928 18:00:16.148038 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:16.276173 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33367
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:33367
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:16.359130 22274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:16.359395 22274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:16.359421 22274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:16.360622 22274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:16.360667 22274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:16.360682 22274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:16.360694 22274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:16.360707 22274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:16.362213 22274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:33367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:16.362465 22274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:16.362659 22274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:16.368381 22280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.368393 22279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.368588 22282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:16.368640 22274 server_base.cc:1034] running on GCE node
I20260928 18:00:16.369007 22274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:16.369344 22274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:16.370520 22274 hybrid_clock.cc:648] HybridClock initialized: now 1790618416370507 us; error 47 us; skew 500 ppm
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:16.373260 22274 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:16.373771 22274 webserver.cc:460] Webserver started at http://127.5.46.126:33287/ using document root <none> and password file <none>
I20260928 18:00:16.374007 22274 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:16.374055 22274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:16.374195 22274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:16.375281 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "53dc89d114cb412ba4b2fe7084e3c9c2"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "79cd2c078dd7e316209f78ec7112a4f1"
server_key_iv: "5389460e0573614f9b9acb850e837cc3"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.375648 22274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "53dc89d114cb412ba4b2fe7084e3c9c2"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "79cd2c078dd7e316209f78ec7112a4f1"
server_key_iv: "5389460e0573614f9b9acb850e837cc3"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.377110 22274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:16.377885 22289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.378069 22274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:16.378168 22274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "53dc89d114cb412ba4b2fe7084e3c9c2"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "79cd2c078dd7e316209f78ec7112a4f1"
server_key_iv: "5389460e0573614f9b9acb850e837cc3"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.378273 22274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:16.406679 22274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:16.409264 22274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:16.409444 22274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:16.413961 22274 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:33367
I20260928 18:00:16.414037 22341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:33367 every 8 connection(s)
I20260928 18:00:16.414426 22274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:16.415328 22342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:16.417728 22342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: Bootstrap starting.
I20260928 18:00:16.418294 22342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:16.418524 22342 log.cc:824] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:16.419314 22342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: No bootstrap required, opened a new log
I20260928 18:00:16.421236 22342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } }
I20260928 18:00:16.421439 22342 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:16.421465 22342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53dc89d114cb412ba4b2fe7084e3c9c2, State: Initialized, Role: FOLLOWER
I20260928 18:00:16.421675 22342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } }
I20260928 18:00:16.421732 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22274
I20260928 18:00:16.421775 22342 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:16.421834 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:16.421816 22342 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:16.421890 22342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:16.422055 5305 external_mini_cluster.cc:1428] Setting key 53e7062da7fdc93c0ab552c65b388edb
I20260928 18:00:16.422874 22342 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } }
I20260928 18:00:16.423015 22342 leader_election.cc:302] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53dc89d114cb412ba4b2fe7084e3c9c2; no voters:
I20260928 18:00:16.423197 22342 leader_election.cc:288] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:16.423218 22347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:16.423360 22347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [term 1 LEADER]: Becoming Leader. State: Replica: 53dc89d114cb412ba4b2fe7084e3c9c2, State: Running, Role: LEADER
I20260928 18:00:16.423486 22342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:16.423492 22347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } }
I20260928 18:00:16.424136 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } } }
I20260928 18:00:16.424139 22349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53dc89d114cb412ba4b2fe7084e3c9c2. Latest consensus state: current_term: 1 leader_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53dc89d114cb412ba4b2fe7084e3c9c2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33367 } } }
I20260928 18:00:16.424253 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:16.424261 22349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:16.424634 22356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:16.425204 22356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:16.426798 22356 catalog_manager.cc:1347] Generated new cluster ID: 02616f206eb24a2bacc790e5917afc84
I20260928 18:00:16.426852 22356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:16.435029 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.422908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60916 (local address 127.5.46.126:33367)
0928 18:00:16.423116 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.423128 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.423166 (+ 38us) server_negotiation.cc:407] Connection header received
0928 18:00:16.423302 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.423309 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.423383 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.424409 (+ 1026us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:16.424542 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.426136 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.427278 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.427954 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.428070 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.430963 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:16.430969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:16.430969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:16.430994 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:16.433214 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.433630 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.433644 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.433645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.433797 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.434005 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.434006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.434006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.434371 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:16.434560 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.434637 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.434790 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":108,"threads_started":1}
I20260928 18:00:16.437561 22356 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:16.438160 22356 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:16.441041 22356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: Generated new TSK 0
I20260928 18:00:16.441242 22356 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 18:00:16.475122 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:16.560575 22368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:16.560813 22368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:16.560842 22368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:16.562168 22368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:16.562224 22368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:16.562258 22368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:16.563982 22368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:16.564292 22368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:16.564541 22368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:16.570111 22376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.570132 22373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.570111 22374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:16.570442 22368 server_base.cc:1034] running on GCE node
I20260928 18:00:16.570686 22368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:16.570951 22368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:16.572124 22368 hybrid_clock.cc:648] HybridClock initialized: now 1790618416572113 us; error 37 us; skew 500 ppm
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:16.575009 22368 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 18:00:16.575512 22368 webserver.cc:460] Webserver started at http://127.5.46.65:33993/ using document root <none> and password file <none>
I20260928 18:00:16.575784 22368 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:16.575881 22368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:16.576036 22368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:16.577127 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "c5852ba41006441bbf98a41809b9dfa1"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "44e9d96aa947d139593f943f4a06867d"
server_key_iv: "58a9badc24efcf4b7b68452a5a02bda5"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.577502 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "c5852ba41006441bbf98a41809b9dfa1"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "44e9d96aa947d139593f943f4a06867d"
server_key_iv: "58a9badc24efcf4b7b68452a5a02bda5"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.578917 22368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:16.579782 22383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.580021 22368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:16.580135 22368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "c5852ba41006441bbf98a41809b9dfa1"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "44e9d96aa947d139593f943f4a06867d"
server_key_iv: "58a9badc24efcf4b7b68452a5a02bda5"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.580238 22368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:16.600567 22368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:16.602795 22368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:16.602974 22368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:16.603250 22368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:16.603588 22368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:16.603642 22368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.603684 22368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:16.603713 22368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.609792 22368 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:37539
I20260928 18:00:16.609905 22496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:37539 every 8 connection(s)
I20260928 18:00:16.610191 22368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 18:00:16.610544 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22368
I20260928 18:00:16.610666 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:16.610910 5305 external_mini_cluster.cc:1428] Setting key 6ec3f340836dfb137315be15602cac57
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
I20260928 18:00:16.620509 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.610975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:33281 (local address 127.5.46.126:33367)
0928 18:00:16.611026 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.611029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.611269 (+ 240us) server_negotiation.cc:407] Connection header received
0928 18:00:16.611945 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.611947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.611962 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.612014 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:16.613003 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.613519 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.614558 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.614788 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.614869 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.617221 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:16.617223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:16.617223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:16.617232 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:16.618940 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.619486 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.619488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.619488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.619539 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.619767 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.619768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.619768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.619979 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:16.620032 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.620364 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.620439 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 18:00:16.620821 22499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.610949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33367 (local address 127.5.46.65:33281)
0928 18:00:16.611135 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.611185 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.611780 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.612097 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.612101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.612695 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:16.612898 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.612903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.613619 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.613620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.614459 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.614461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.614877 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.614878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.615022 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.615613 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:16.615622 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:16.617117 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:16.619048 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.619056 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.619058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.619388 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.619607 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.619608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.619608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.619724 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.620052 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:16.620060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:16.620232 (+ 172us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.620335 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.620481 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260928 18:00:16.621506 22497 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
I20260928 18:00:16.621616 22497 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:16.621850 22497 heartbeater.cc:511] Master 127.5.46.126:33367 requested a full tablet report, sending...
I20260928 18:00:16.622486 22306 ts_manager.cc:194] Registered new tserver with Master: c5852ba41006441bbf98a41809b9dfa1 (127.5.46.65:37539)
I20260928 18:00:16.623332 22306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:33281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 18:00:16.639500 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:16.722651 22502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:16.722878 22502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:16.722903 22502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:16.724249 22502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:16.724296 22502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:16.724324 22502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:16.725975 22502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:16.726244 22502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:16.726500 22502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:16.731644 22510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.731629 22507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.731704 22508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:16.732125 22502 server_base.cc:1034] running on GCE node
I20260928 18:00:16.732347 22502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:16.732659 22502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:16.733896 22502 hybrid_clock.cc:648] HybridClock initialized: now 1790618416733888 us; error 42 us; skew 500 ppm
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:16.736821 22502 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 18:00:16.737404 22502 webserver.cc:460] Webserver started at http://127.5.46.66:38303/ using document root <none> and password file <none>
I20260928 18:00:16.737670 22502 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:16.737742 22502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:16.737900 22502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:16.739029 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "9545e143e2bb4bbdbbfe8898a7176d6a"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "c46fc968275dd9655d5489002d424471"
server_key_iv: "63a756b433677546a749b7a1bc1df0a9"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.739418 22502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "9545e143e2bb4bbdbbfe8898a7176d6a"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "c46fc968275dd9655d5489002d424471"
server_key_iv: "63a756b433677546a749b7a1bc1df0a9"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.741123 22502 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 18:00:16.742043 22517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.742328 22502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:16.742487 22502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "9545e143e2bb4bbdbbfe8898a7176d6a"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "c46fc968275dd9655d5489002d424471"
server_key_iv: "63a756b433677546a749b7a1bc1df0a9"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.742579 22502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:16.768914 22502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:16.771462 22502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:16.771726 22502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:16.772042 22502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:16.772426 22502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:16.772485 22502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.772533 22502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:16.772562 22502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.779179 22502 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:34259
I20260928 18:00:16.779233 22630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:34259 every 8 connection(s)
I20260928 18:00:16.779630 22502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:16.785320 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22502
I20260928 18:00:16.785454 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:16.785677 5305 external_mini_cluster.cc:1428] Setting key ee45e3420d77f34f777ea32a07686e5b
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:16.790831 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.780515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48255 (local address 127.5.46.126:33367)
0928 18:00:16.780623 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.780625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.780835 (+ 210us) server_negotiation.cc:407] Connection header received
0928 18:00:16.781359 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.781360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.781372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.781434 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:16.782453 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.783023 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.784213 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.784409 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.784472 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.787468 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:16.787469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:16.787469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:16.787476 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:16.789050 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.789542 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.789544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.789545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.789614 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.789969 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.789971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.789971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.790134 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:16.790209 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.790589 (+ 380us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.790697 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 18:00:16.790928 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.780471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33367 (local address 127.5.46.66:48255)
0928 18:00:16.780675 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.780749 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.781267 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.781496 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.781499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.782106 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:16.782310 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.782315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.783165 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.783166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.784127 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.784130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.784508 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.784509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.784623 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.785311 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:16.785320 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:16.787370 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:16.789157 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.789170 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.789172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.789458 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.789757 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.789758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.789758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.789887 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.790227 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:16.790233 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:16.790385 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.790541 (+ 156us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.790711 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":110,"threads_started":1}
I20260928 18:00:16.791544 22631 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33367
I20260928 18:00:16.791643 22631 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:16.792044 22631 heartbeater.cc:511] Master 127.5.46.126:33367 requested a full tablet report, sending...
I20260928 18:00:16.792549 22306 ts_manager.cc:194] Registered new tserver with Master: 9545e143e2bb4bbdbbfe8898a7176d6a (127.5.46.66:34259)
I20260928 18:00:16.793102 22306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:48255
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 18:00:16.815480 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:46569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:16.898623 22636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:16.898850 22636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:16.898878 22636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:16.900061 22636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:16.900105 22636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:16.900132 22636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:16.901628 22636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:46569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:33367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:16.901851 22636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:16.902045 22636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:16.907449 22644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:16.907524 22641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:16.907680 22636 server_base.cc:1034] running on GCE node
W20260928 18:00:16.907538 22642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:16.908033 22636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:16.908296 22636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:16.909467 22636 hybrid_clock.cc:648] HybridClock initialized: now 1790618416909463 us; error 37 us; skew 500 ppm
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:16.912144 22636 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 18:00:16.912662 22636 webserver.cc:460] Webserver started at http://127.5.46.67:38057/ using document root <none> and password file <none>
I20260928 18:00:16.912902 22636 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:16.912971 22636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:16.913122 22636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:16.914214 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "6b1b5b0ed4287188231ed60d31a52c36"
server_key_iv: "97d2b99d6ff2c8b77f8d5b51b9715a58"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.914585 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "6b1b5b0ed4287188231ed60d31a52c36"
server_key_iv: "97d2b99d6ff2c8b77f8d5b51b9715a58"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.916043 22636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 18:00:16.916853 22651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.917060 22636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:16.917168 22636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176"
format_stamp: "Formatted at 2026-09-28 18:00:16 on dist-test-slave-z2t3"
server_key: "6b1b5b0ed4287188231ed60d31a52c36"
server_key_iv: "97d2b99d6ff2c8b77f8d5b51b9715a58"
server_key_version: "encryptionkey@0"
I20260928 18:00:16.917268 22636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:16.933771 22636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:16.936367 22636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:16.936568 22636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:16.936830 22636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:16.937156 22636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:16.937224 22636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.937275 22636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:16.937306 22636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:16.943960 22636 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45655
I20260928 18:00:16.944034 22764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45655 every 8 connection(s)
I20260928 18:00:16.944408 22636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:16.950851 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22636
I20260928 18:00:16.950974 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:16.951174 5305 external_mini_cluster.cc:1428] Setting key 41317124fe025ba20934fc271b8f061c
Sep 28 18:00:16 dist-test-slave-z2t3 krb5kdc[22261](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618416, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:16.955216 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.945221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:45579 (local address 127.5.46.126:33367)
0928 18:00:16.945272 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.945274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.945516 (+ 242us) server_negotiation.cc:407] Connection header received
0928 18:00:16.946191 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.946192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.946204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.946269 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:16.947336 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.947920 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.949130 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.949331 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.949392 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.952181 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:16.952183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:16.952183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:16.952192 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:16.953736 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.954185 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.954187 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.954187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.954249 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.954526 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.954526 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.954527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.954653 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:16.954730 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.955032 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.955120 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:16.955446 22767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.945206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:33367 (local address 127.5.46.67:45579)
0928 18:00:16.945375 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.945413 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.946047 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.946330 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.946333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.946942 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:16.947213 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.947219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.948026 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.948028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.948982 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.948984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.949421 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.949422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.949546 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.950141 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:16.950151 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:16.952080 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:16.953850 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.953858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.953861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.954102 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.954320 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:16.954321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:16.954322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:16.954449 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:16.954732 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:16.954738 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:16.954891 (+ 153us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.954995 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.955178 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260928 18:00:16.956080 22765 heartbeater.cc:347] Connected to a master server at 127.5.46.126:33367
I20260928 18:00:16.956195 22765 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:16.956418 22765 heartbeater.cc:511] Master 127.5.46.126:33367 requested a full tablet report, sending...
I20260928 18:00:16.956868 22306 ts_manager.cc:194] Registered new tserver with Master: 32a8cadeb94c4c8abd28dd1fb5bb1176 (127.5.46.67:45655)
I20260928 18:00:16.957291 22306 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:45579
I20260928 18:00:16.963382 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:16.971099 22346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.964819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.5.46.126:33367)
0928 18:00:16.964895 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.964897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.964976 (+ 79us) server_negotiation.cc:407] Connection header received
0928 18:00:16.965018 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.965019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.965028 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.965076 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:16.965822 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.966350 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.967090 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.967282 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.967337 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.968565 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:16.968566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:16.968567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:16.968573 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:16.970042 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.970369 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.970370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.970370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.970423 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:16.970584 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:16.970585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:16.970585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:16.970774 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:16.970852 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.970943 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.971005 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 18:00:16.972962 22306 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:16.974527 22306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:16.981770 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.5.46.66:34259)
0928 18:00:16.977774 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.977779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.977788 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:16.977997 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.978002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.978021 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.978263 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:16.978322 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.978976 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.980571 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.981379 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.981478 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.981545 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.981640 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.981678 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260928 18:00:16.981769 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:34259 (local address 127.0.0.1:34972)
0928 18:00:16.977523 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.977532 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.977572 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.978128 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.978129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.978131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:16.978187 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.978189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.979105 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.979106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.980450 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.980454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.981499 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.981500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.981561 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.981566 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.981617 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.981679 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260928 18:00:16.982013 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.5.46.65:37539)
0928 18:00:16.977677 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.977681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.977693 (+ 12us) server_negotiation.cc:407] Connection header received
0928 18:00:16.977759 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.977761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.977776 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.978075 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:16.978155 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.979280 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.980576 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.981322 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.981794 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.981868 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.981895 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.981935 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":83,"threads_started":1}
I20260928 18:00:16.982041 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:37539 (local address 127.0.0.1:44956)
0928 18:00:16.977319 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.977361 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.977423 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.977908 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.977908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.977910 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:16.977987 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.977989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.979396 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.979397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.980470 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.980473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.981432 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.981433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.981512 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.981525 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.981574 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.981658 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260928 18:00:16.982884 22431 tablet_service.cc:1467] Processing CreateTablet for tablet b1b8b09415924183973d882ee2c5c1fb (DEFAULT_TABLE table=test-table [id=5e609047cfd24c2cb259f80d6975db25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:16.983009 22565 tablet_service.cc:1467] Processing CreateTablet for tablet b1b8b09415924183973d882ee2c5c1fb (DEFAULT_TABLE table=test-table [id=5e609047cfd24c2cb259f80d6975db25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:16.983270 22431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b8b09415924183973d882ee2c5c1fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:16.983387 22565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b8b09415924183973d882ee2c5c1fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:16.984232 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:45655 (local address 127.0.0.1:58940)
0928 18:00:16.977478 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:16.977488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:16.977549 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:16.977888 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:16.977891 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:16.977893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:16.977987 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.977990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.980597 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.980598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.981280 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:16.981282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.983934 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.983935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:16.984037 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.984042 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:16.984107 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 18:00:16.984150 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":9,"thread_start_us":69,"threads_started":1}
I20260928 18:00:16.984266 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:16.977232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.5.46.67:45655)
0928 18:00:16.977633 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:16.977638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:16.977654 (+ 16us) server_negotiation.cc:407] Connection header received
0928 18:00:16.977707 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:16.977710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:16.977728 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:16.977858 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:16.978087 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.979288 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.981433 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:16.982549 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:16.982614 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:16.982667 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:16.984138 (+ 1471us) server_negotiation.cc:299] Negotiation successful
0928 18:00:16.984186 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":103,"threads_started":1}
I20260928 18:00:16.985117 22699 tablet_service.cc:1467] Processing CreateTablet for tablet b1b8b09415924183973d882ee2c5c1fb (DEFAULT_TABLE table=test-table [id=5e609047cfd24c2cb259f80d6975db25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:16.985500 22699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1b8b09415924183973d882ee2c5c1fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:16.986464 22782 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Bootstrap starting.
I20260928 18:00:16.986792 22783 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: Bootstrap starting.
I20260928 18:00:16.987181 22782 tablet_bootstrap.cc:654] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:16.987432 22783 tablet_bootstrap.cc:654] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:16.987820 22782 log.cc:824] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:16.988068 22783 log.cc:824] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:16.988444 22786 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Bootstrap starting.
I20260928 18:00:16.988615 22782 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: No bootstrap required, opened a new log
I20260928 18:00:16.988687 22782 ts_tablet_manager.cc:1397] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:16.989099 22786 tablet_bootstrap.cc:654] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:16.989701 22786 log.cc:824] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:16.990520 22786 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: No bootstrap required, opened a new log
I20260928 18:00:16.990592 22786 ts_tablet_manager.cc:1397] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:16.990906 22782 raft_consensus.cc:348] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.991096 22782 raft_consensus.cc:374] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:16.991134 22782 raft_consensus.cc:729] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9545e143e2bb4bbdbbfe8898a7176d6a, State: Initialized, Role: FOLLOWER
I20260928 18:00:16.991261 22782 consensus_queue.cc:260] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.991559 22782 ts_tablet_manager.cc:1428] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:16.991590 22631 heartbeater.cc:503] Master 127.5.46.126:33367 was elected leader, sending a full tablet report...
I20260928 18:00:16.992326 22783 tablet_bootstrap.cc:492] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: No bootstrap required, opened a new log
I20260928 18:00:16.992422 22783 ts_tablet_manager.cc:1397] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260928 18:00:16.993808 22783 raft_consensus.cc:348] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.993978 22783 raft_consensus.cc:374] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:16.994035 22783 raft_consensus.cc:729] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5852ba41006441bbf98a41809b9dfa1, State: Initialized, Role: FOLLOWER
I20260928 18:00:16.994170 22783 consensus_queue.cc:260] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.994479 22783 ts_tablet_manager.cc:1428] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 18:00:16.994509 22497 heartbeater.cc:503] Master 127.5.46.126:33367 was elected leader, sending a full tablet report...
I20260928 18:00:16.994813 22786 raft_consensus.cc:348] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.994979 22786 raft_consensus.cc:374] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:16.995015 22786 raft_consensus.cc:729] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32a8cadeb94c4c8abd28dd1fb5bb1176, State: Initialized, Role: FOLLOWER
I20260928 18:00:16.995131 22786 consensus_queue.cc:260] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:16.995414 22786 ts_tablet_manager.cc:1428] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260928 18:00:16.995431 22765 heartbeater.cc:503] Master 127.5.46.126:33367 was elected leader, sending a full tablet report...
W20260928 18:00:17.030333 22632 tablet.cc:2369] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 18:00:17.068054 22788 raft_consensus.cc:482] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:17.068207 22788 raft_consensus.cc:504] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:17.068557 22788 leader_election.cc:288] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32a8cadeb94c4c8abd28dd1fb5bb1176 (127.5.46.67:45655), c5852ba41006441bbf98a41809b9dfa1 (127.5.46.65:37539)
I20260928 18:00:17.071646 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.068971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:50197 (local address 127.5.46.67:45655)
0928 18:00:17.069029 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.069032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.069063 (+ 31us) server_negotiation.cc:407] Connection header received
0928 18:00:17.069141 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.069142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.069153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.069221 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.069426 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.069932 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.070903 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.071283 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.071370 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.071381 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.071539 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.071580 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 18:00:17.071658 22791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.068774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:45655 (local address 127.5.46.66:50197)
0928 18:00:17.069002 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.069013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.069075 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.069240 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.069240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.069242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.069348 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.069349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.070026 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.070028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.070805 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.070807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.071371 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.071372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.071447 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.071451 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.071522 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.071565 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":17,"thread_start_us":143,"threads_started":1}
I20260928 18:00:17.071751 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.068949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:39871 (local address 127.5.46.65:37539)
0928 18:00:17.069003 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.069005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.069014 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:17.069044 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.069045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.069058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.069146 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.069330 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.069853 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.070911 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.071347 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.071445 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.071457 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.071628 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.071674 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 18:00:17.071751 22633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.068774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:37539 (local address 127.5.46.66:39871)
0928 18:00:17.068832 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.068847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.068926 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.069153 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.069154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.069156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.069245 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.069247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.069955 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.069956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.070801 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.070803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.071445 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.071447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.071541 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.071544 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.071621 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.071675 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 18:00:17.072016 22719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b8b09415924183973d882ee2c5c1fb" candidate_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" is_pre_election: true
I20260928 18:00:17.072172 22719 raft_consensus.cc:2454] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9545e143e2bb4bbdbbfe8898a7176d6a in term 0.
I20260928 18:00:17.072198 22451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b8b09415924183973d882ee2c5c1fb" candidate_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5852ba41006441bbf98a41809b9dfa1" is_pre_election: true
I20260928 18:00:17.072372 22451 raft_consensus.cc:2454] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9545e143e2bb4bbdbbfe8898a7176d6a in term 0.
I20260928 18:00:17.072417 22521 leader_election.cc:302] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a8cadeb94c4c8abd28dd1fb5bb1176, 9545e143e2bb4bbdbbfe8898a7176d6a; no voters:
I20260928 18:00:17.072542 22788 raft_consensus.cc:2781] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:17.072603 22788 raft_consensus.cc:482] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:17.072657 22788 raft_consensus.cc:3037] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.073550 22788 raft_consensus.cc:504] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:17.073698 22788 leader_election.cc:288] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [CANDIDATE]: Term 1 election: Requested vote from peers 32a8cadeb94c4c8abd28dd1fb5bb1176 (127.5.46.67:45655), c5852ba41006441bbf98a41809b9dfa1 (127.5.46.65:37539)
I20260928 18:00:17.073904 22719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b8b09415924183973d882ee2c5c1fb" candidate_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176"
I20260928 18:00:17.073886 22451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1b8b09415924183973d882ee2c5c1fb" candidate_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5852ba41006441bbf98a41809b9dfa1"
I20260928 18:00:17.073985 22719 raft_consensus.cc:3037] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.073985 22451 raft_consensus.cc:3037] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.074863 22451 raft_consensus.cc:2454] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9545e143e2bb4bbdbbfe8898a7176d6a in term 1.
I20260928 18:00:17.074863 22719 raft_consensus.cc:2454] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9545e143e2bb4bbdbbfe8898a7176d6a in term 1.
I20260928 18:00:17.075088 22521 leader_election.cc:302] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32a8cadeb94c4c8abd28dd1fb5bb1176, 9545e143e2bb4bbdbbfe8898a7176d6a; no voters:
I20260928 18:00:17.075213 22788 raft_consensus.cc:2781] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:17.075398 22788 raft_consensus.cc:686] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 1 LEADER]: Becoming Leader. State: Replica: 9545e143e2bb4bbdbbfe8898a7176d6a, State: Running, Role: LEADER
I20260928 18:00:17.075484 22788 consensus_queue.cc:237] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } }
I20260928 18:00:17.076360 22306 catalog_manager.cc:5495] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9545e143e2bb4bbdbbfe8898a7176d6a (127.5.46.66). New cstate: current_term: 1 leader_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9545e143e2bb4bbdbbfe8898a7176d6a" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 34259 } health_report { overall_health: HEALTHY } } }
I20260928 18:00:17.101506 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.099030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.5.46.66:34259)
0928 18:00:17.099100 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.099102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.099110 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:17.099166 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.099167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.099187 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.099255 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:17.099440 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.099993 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.100792 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.100988 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.101060 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.101211 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:17.101316 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:17.101381 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.101418 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.101437 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 18:00:17.103571 22451 raft_consensus.cc:1261] T b1b8b09415924183973d882ee2c5c1fb P c5852ba41006441bbf98a41809b9dfa1 [term 1 FOLLOWER]: Refusing update from remote peer 9545e143e2bb4bbdbbfe8898a7176d6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:17.103693 22719 raft_consensus.cc:1261] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 1 FOLLOWER]: Refusing update from remote peer 9545e143e2bb4bbdbbfe8898a7176d6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:17.103976 22788 consensus_queue.cc:1035] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5852ba41006441bbf98a41809b9dfa1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 37539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:17.104151 22792 consensus_queue.cc:1035] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "32a8cadeb94c4c8abd28dd1fb5bb1176" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:17.106501 22795 mvcc.cc:205] Tried to move back new op lower bound from 7334373036452990976 to 7334373036341796864. Current Snapshot: MvccSnapshot[applied={T|T < 7334373036452990976}]
I20260928 18:00:17.106662 22796 mvcc.cc:205] Tried to move back new op lower bound from 7334373036452990976 to 7334373036341796864. Current Snapshot: MvccSnapshot[applied={T|T < 7334373036452990976}]
I20260928 18:00:17.107417 22797 mvcc.cc:205] Tried to move back new op lower bound from 7334373036452990976 to 7334373036341796864. Current Snapshot: MvccSnapshot[applied={T|T < 7334373036452990976}]
I20260928 18:00:17.109069 22305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:17.109179 22305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:17.110132 22305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53dc89d114cb412ba4b2fe7084e3c9c2: Sending DeleteTablet for 3 replicas of tablet b1b8b09415924183973d882ee2c5c1fb
I20260928 18:00:17.110523 22431 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b8b09415924183973d882ee2c5c1fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:44956
I20260928 18:00:17.110534 22699 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b8b09415924183973d882ee2c5c1fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:58940
I20260928 18:00:17.110536 22565 tablet_service.cc:1514] Processing DeleteTablet for tablet b1b8b09415924183973d882ee2c5c1fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:34972
I20260928 18:00:17.110786 22806 tablet_replica.cc:330] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: stopping tablet replica
I20260928 18:00:17.110891 22806 raft_consensus.cc:2229] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:17.111001 22806 raft_consensus.cc:2258] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:17.111207 22807 tablet_replica.cc:330] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: stopping tablet replica
I20260928 18:00:17.111312 22807 raft_consensus.cc:2229] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:17.111428 22807 raft_consensus.cc:2258] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:17.111466 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22368
I20260928 18:00:17.112999 22806 ts_tablet_manager.cc:1905] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:17.113075 22807 ts_tablet_manager.cc:1905] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:17.115137 22806 ts_tablet_manager.cc:1918] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:17.115126 22807 ts_tablet_manager.cc:1918] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:17.115213 22807 log.cc:1196] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/b1b8b09415924183973d882ee2c5c1fb
I20260928 18:00:17.115338 22806 log.cc:1196] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/b1b8b09415924183973d882ee2c5c1fb
I20260928 18:00:17.115541 22807 ts_tablet_manager.cc:1939] T b1b8b09415924183973d882ee2c5c1fb P 32a8cadeb94c4c8abd28dd1fb5bb1176: Deleting consensus metadata
I20260928 18:00:17.115751 22806 ts_tablet_manager.cc:1939] T b1b8b09415924183973d882ee2c5c1fb P 9545e143e2bb4bbdbbfe8898a7176d6a: Deleting consensus metadata
I20260928 18:00:17.116173 22293 catalog_manager.cc:4841] TS 32a8cadeb94c4c8abd28dd1fb5bb1176 (127.5.46.67:45655): tablet b1b8b09415924183973d882ee2c5c1fb (table test-table [id=5e609047cfd24c2cb259f80d6975db25]) successfully deleted
I20260928 18:00:17.116223 22290 catalog_manager.cc:4841] TS 9545e143e2bb4bbdbbfe8898a7176d6a (127.5.46.66:34259): tablet b1b8b09415924183973d882ee2c5c1fb (table test-table [id=5e609047cfd24c2cb259f80d6975db25]) successfully deleted
W20260928 18:00:17.118705 22290 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:37539 (error 108)
I20260928 18:00:17.118880 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22502
I20260928 18:00:17.120098 22779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.119944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:37539 (local address 127.0.0.1:44968)
0928 18:00:17.120008 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.120051 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 18:00:17.120275 22290 catalog_manager.cc:4584] TS c5852ba41006441bbf98a41809b9dfa1 (127.5.46.65:37539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1b8b09415924183973d882ee2c5c1fb: Network error: Client connection negotiation failed: client connection to 127.5.46.65:37539: connect: Connection refused (error 111)
I20260928 18:00:17.124284 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22636
I20260928 18:00:17.129181 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22274
2026-09-28T18:00:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2020 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T18:00:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:17Z Disabled control of system clock
I20260928 18:00:17.157426 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39783
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:39783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:17.240084 22812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:17.240337 22812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:17.240360 22812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:17.241724 22812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:17.241775 22812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:17.241787 22812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:17.241801 22812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:17.241813 22812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:17.243274 22812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:39783
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:39783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:17.243569 22812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:17.243942 22812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:17.248960 22820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.248986 22817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.248991 22818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:17.249189 22812 server_base.cc:1034] running on GCE node
I20260928 18:00:17.249547 22812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:17.249854 22812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:17.251026 22812 hybrid_clock.cc:648] HybridClock initialized: now 1790618417251015 us; error 34 us; skew 500 ppm
I20260928 18:00:17.252540 22812 webserver.cc:460] Webserver started at http://127.5.46.126:45195/ using document root <none> and password file <none>
I20260928 18:00:17.252823 22812 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:17.252900 22812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:17.253049 22812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:17.254132 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "d0f70d1e0c0042a2840c8315e717b740"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "ee2bfc2b363f15092b95948bdc83b09a"
server_key_iv: "0e32e6f8e9d06ff3ff3b90bcdc4b3c70"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.254590 22812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "d0f70d1e0c0042a2840c8315e717b740"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "ee2bfc2b363f15092b95948bdc83b09a"
server_key_iv: "0e32e6f8e9d06ff3ff3b90bcdc4b3c70"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.255992 22812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260928 18:00:17.256771 22826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.256945 22812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:17.257051 22812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "d0f70d1e0c0042a2840c8315e717b740"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "ee2bfc2b363f15092b95948bdc83b09a"
server_key_iv: "0e32e6f8e9d06ff3ff3b90bcdc4b3c70"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.257126 22812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:17.283778 22812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:17.284178 22812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:17.284346 22812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:17.288507 22812 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:39783
I20260928 18:00:17.288563 22878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:39783 every 8 connection(s)
I20260928 18:00:17.288951 22812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:17.289964 22879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:17.292439 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22812
I20260928 18:00:17.292585 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:17.292615 22879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: Bootstrap starting.
I20260928 18:00:17.292835 5305 external_mini_cluster.cc:1428] Setting key c401d6011c153f2301bfbea1f6a99ab0
I20260928 18:00:17.293337 22879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:17.293756 22879 log.cc:824] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:17.294559 22879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: No bootstrap required, opened a new log
I20260928 18:00:17.296622 22879 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } }
I20260928 18:00:17.296808 22879 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:17.296852 22879 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f70d1e0c0042a2840c8315e717b740, State: Initialized, Role: FOLLOWER
I20260928 18:00:17.296988 22879 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } }
I20260928 18:00:17.297070 22879 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:17.297134 22879 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:17.297183 22879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.297897 22879 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } }
I20260928 18:00:17.298003 22879 leader_election.cc:302] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f70d1e0c0042a2840c8315e717b740; no voters:
I20260928 18:00:17.298210 22879 leader_election.cc:288] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:17.298331 22884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:17.298517 22884 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [term 1 LEADER]: Becoming Leader. State: Replica: d0f70d1e0c0042a2840c8315e717b740, State: Running, Role: LEADER
I20260928 18:00:17.298579 22879 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:17.299218 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.293441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.5.46.126:39783)
0928 18:00:17.293687 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.293695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.293722 (+ 27us) server_negotiation.cc:407] Connection header received
0928 18:00:17.294208 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.294215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.294282 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.294409 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:17.294714 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.295691 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.296909 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.297486 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.297588 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.298401 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:17.298406 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:17.298406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:17.298635 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:17.298679 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:17.298695 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:17.298761 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.298886 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.298982 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
I20260928 18:00:17.299549 22884 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } }
I20260928 18:00:17.300107 22885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f70d1e0c0042a2840c8315e717b740. Latest consensus state: current_term: 1 leader_uuid: "d0f70d1e0c0042a2840c8315e717b740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } } }
I20260928 18:00:17.300293 22886 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0f70d1e0c0042a2840c8315e717b740" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f70d1e0c0042a2840c8315e717b740" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 39783 } } }
I20260928 18:00:17.300376 22885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:17.300382 22886 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740 [sys.catalog]: This master's current role is: LEADER
W20260928 18:00:17.300511 22898 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:17.300613 22898 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:17.300738 22900 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:17.300941 22900 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:17.302531 22900 catalog_manager.cc:1347] Generated new cluster ID: 6d5e233ba3024ce8be4014c4cdbc8d69
I20260928 18:00:17.302604 22900 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:17.320024 22900 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:17.320653 22900 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:17.327888 22900 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: Generated new TSK 0
I20260928 18:00:17.328137 22900 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:17.330541 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--builtin_ntp_servers=127.5.46.84:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:17.420282 22903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:17.420543 22903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:17.420583 22903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:17.421854 22903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:17.421928 22903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:17.421972 22903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:17.423432 22903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:17.423740 22903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:17.424144 22903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:17.430393 22908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.430393 22909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.430462 22911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:17.430459 22903 server_base.cc:1034] running on GCE node
I20260928 18:00:17.430882 22903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:17.431169 22903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:17.432338 22903 hybrid_clock.cc:648] HybridClock initialized: now 1790618417432297 us; error 62 us; skew 500 ppm
I20260928 18:00:17.433730 22903 webserver.cc:460] Webserver started at http://127.5.46.65:33173/ using document root <none> and password file <none>
I20260928 18:00:17.433991 22903 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:17.434039 22903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:17.434175 22903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:17.435307 22903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "a8a418aeea1b4a928e5feb5708ba99d6"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c391d5ae159b9218e81cbb8891117081"
server_key_iv: "fbe7281da3e526812bb92dd7f04d3068"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.435676 22903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "a8a418aeea1b4a928e5feb5708ba99d6"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c391d5ae159b9218e81cbb8891117081"
server_key_iv: "fbe7281da3e526812bb92dd7f04d3068"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.437090 22903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:17.438045 22917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.438325 22903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:17.438443 22903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "a8a418aeea1b4a928e5feb5708ba99d6"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c391d5ae159b9218e81cbb8891117081"
server_key_iv: "fbe7281da3e526812bb92dd7f04d3068"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.438530 22903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:17.459443 22903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:17.459928 22903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:17.460115 22903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:17.460391 22903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:17.460750 22903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:17.460808 22903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.460844 22903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:17.460871 22903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.467077 22903 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:45333
I20260928 18:00:17.467162 23030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:45333 every 8 connection(s)
I20260928 18:00:17.467566 22903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 18:00:17.469357 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 22903
I20260928 18:00:17.469486 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:17.469710 5305 external_mini_cluster.cc:1428] Setting key e9bbff843fb1b832c23691a2bb3b5aab
I20260928 18:00:17.470886 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--builtin_ntp_servers=127.5.46.84:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 18:00:17.474431 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.468461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:56259 (local address 127.5.46.126:39783)
0928 18:00:17.468529 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.468531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.468807 (+ 276us) server_negotiation.cc:407] Connection header received
0928 18:00:17.469305 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.469307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.469326 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.469381 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:17.470307 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.471222 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.472620 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.472835 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.472894 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.474122 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:17.474124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:17.474124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:17.474130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:17.474157 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:17.474162 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:17.474241 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.474314 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.474345 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 18:00:17.474679 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.468423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:39783 (local address 127.5.46.65:56259)
0928 18:00:17.468644 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.468687 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.469182 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.469458 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.469462 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.469477 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:17.470216 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.470224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.471344 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.471345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.472491 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.472495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.472938 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.472939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.473119 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.473890 (+ 771us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:17.473890 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:17.473924 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:17.473950 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:17.474234 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:17.474242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:17.474245 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.474320 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.474439 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":103,"threads_started":1}
I20260928 18:00:17.475471 23031 heartbeater.cc:347] Connected to a master server at 127.5.46.126:39783
I20260928 18:00:17.475607 23031 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:17.475896 23031 heartbeater.cc:511] Master 127.5.46.126:39783 requested a full tablet report, sending...
I20260928 18:00:17.476445 22843 ts_manager.cc:194] Registered new tserver with Master: a8a418aeea1b4a928e5feb5708ba99d6 (127.5.46.65:45333)
I20260928 18:00:17.477101 22843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.65:56259
W20260928 18:00:17.557066 23034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:17.557286 23034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:17.557327 23034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:17.558559 23034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:17.558609 23034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:17.558643 23034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:17.560397 23034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:17.560711 23034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:17.561057 23034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:17.566427 23039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.566421 23040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.566427 23042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:17.566617 23034 server_base.cc:1034] running on GCE node
I20260928 18:00:17.566895 23034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:17.567174 23034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:17.568343 23034 hybrid_clock.cc:648] HybridClock initialized: now 1790618417568329 us; error 30 us; skew 500 ppm
I20260928 18:00:17.569844 23034 webserver.cc:460] Webserver started at http://127.5.46.66:33315/ using document root <none> and password file <none>
I20260928 18:00:17.570120 23034 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:17.570197 23034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:17.570359 23034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:17.571475 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "4996bc2ff2a245ea866bceffb617f029"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c81c97a8aa5c445bd2f4b8af43bafbb9"
server_key_iv: "8916f17c598b41ab87b8afed6f38d4ec"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.571913 23034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "4996bc2ff2a245ea866bceffb617f029"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c81c97a8aa5c445bd2f4b8af43bafbb9"
server_key_iv: "8916f17c598b41ab87b8afed6f38d4ec"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.573305 23034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:17.574106 23048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.574280 23034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 18:00:17.574379 23034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "4996bc2ff2a245ea866bceffb617f029"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "c81c97a8aa5c445bd2f4b8af43bafbb9"
server_key_iv: "8916f17c598b41ab87b8afed6f38d4ec"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.574465 23034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:17.606496 23034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:17.606868 23034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:17.607035 23034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:17.607286 23034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:17.607627 23034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:17.607683 23034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.607758 23034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:17.607787 23034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.613700 23034 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:38513
I20260928 18:00:17.613734 23161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:38513 every 8 connection(s)
I20260928 18:00:17.614102 23034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:17.617161 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23034
I20260928 18:00:17.617278 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:17.617458 5305 external_mini_cluster.cc:1428] Setting key e236bd8280766e71f8de92856990d193
I20260928 18:00:17.618404 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--builtin_ntp_servers=127.5.46.84:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 18:00:17.619747 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.614871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:40631 (local address 127.5.46.126:39783)
0928 18:00:17.614924 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.614927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.615234 (+ 307us) server_negotiation.cc:407] Connection header received
0928 18:00:17.615634 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.615636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.615647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.615707 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:17.616391 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.616914 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.618142 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.618367 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.618447 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.619392 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:17.619393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:17.619394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:17.619399 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:17.619415 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:17.619419 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:17.619473 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.619625 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.619658 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 18:00:17.620031 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.614822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:39783 (local address 127.5.46.66:40631)
0928 18:00:17.615012 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.615056 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.615542 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.615768 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.615771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.615779 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:17.616301 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.616306 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.617037 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.617038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.618074 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.618077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.618455 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.618456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.618607 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.619243 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:17.619244 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:17.619274 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:17.619296 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:17.619502 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:17.619510 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:17.619514 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.619613 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.619737 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260928 18:00:17.620707 23162 heartbeater.cc:347] Connected to a master server at 127.5.46.126:39783
I20260928 18:00:17.620823 23162 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:17.621011 23162 heartbeater.cc:511] Master 127.5.46.126:39783 requested a full tablet report, sending...
I20260928 18:00:17.621448 22843 ts_manager.cc:194] Registered new tserver with Master: 4996bc2ff2a245ea866bceffb617f029 (127.5.46.66:38513)
I20260928 18:00:17.621977 22843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.66:40631
W20260928 18:00:17.700811 23165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:17.701052 23165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:17.701087 23165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:17.702245 23165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:17.702308 23165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:17.702365 23165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:17.703760 23165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:39783
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:17.704068 23165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:17.704380 23165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:17.709465 23170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.709461 23171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:17.709493 23173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:17.709659 23165 server_base.cc:1034] running on GCE node
I20260928 18:00:17.709854 23165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:17.710058 23165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:17.711261 23165 hybrid_clock.cc:648] HybridClock initialized: now 1790618417711259 us; error 41 us; skew 500 ppm
I20260928 18:00:17.712735 23165 webserver.cc:460] Webserver started at http://127.5.46.67:46153/ using document root <none> and password file <none>
I20260928 18:00:17.713052 23165 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:17.713130 23165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:17.713284 23165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:17.714421 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "7ef7f061891741c78e9cbfee7dc62756"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "cd1977b5e5a9bea41511eda6f3aeb889"
server_key_iv: "a624119422cd50bae33c7f8ec417f8b6"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.714725 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "7ef7f061891741c78e9cbfee7dc62756"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "cd1977b5e5a9bea41511eda6f3aeb889"
server_key_iv: "a624119422cd50bae33c7f8ec417f8b6"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.716166 23165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:17.716980 23179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.717180 23165 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:17.717289 23165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "7ef7f061891741c78e9cbfee7dc62756"
format_stamp: "Formatted at 2026-09-28 18:00:17 on dist-test-slave-z2t3"
server_key: "cd1977b5e5a9bea41511eda6f3aeb889"
server_key_iv: "a624119422cd50bae33c7f8ec417f8b6"
server_key_version: "encryptionkey@0"
I20260928 18:00:17.717374 23165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:17.761825 23165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:17.762200 23165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:17.762387 23165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:17.762653 23165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:17.763022 23165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:17.763085 23165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.763130 23165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:17.763157 23165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:17.769337 23165 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:32815
I20260928 18:00:17.769383 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:32815 every 8 connection(s)
I20260928 18:00:17.769733 23165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:17.774387 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23165
I20260928 18:00:17.774528 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:17.774748 5305 external_mini_cluster.cc:1428] Setting key e7335d9fcf83948e3f3bc78cd98492a3
I20260928 18:00:17.775382 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.770456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34535 (local address 127.5.46.126:39783)
0928 18:00:17.770504 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.770506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.770766 (+ 260us) server_negotiation.cc:407] Connection header received
0928 18:00:17.771279 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.771280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.771290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.771354 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:17.772249 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.772743 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.773798 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.773991 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.774055 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.775072 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:17.775073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:17.775074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:17.775079 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:17.775098 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:17.775103 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:17.775162 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.775268 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.775298 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:17.775638 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.770420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:39783 (local address 127.5.46.67:34535)
0928 18:00:17.770606 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.770649 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.771188 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.771472 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.771477 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.771497 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:17.772147 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.772153 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.772853 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.772854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.773673 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.773675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.774075 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.774076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.774195 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.774880 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:17.774881 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:17.774928 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:17.774959 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:17.775175 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:17.775181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:17.775184 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.775264 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.775383 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260928 18:00:17.776079 23293 heartbeater.cc:347] Connected to a master server at 127.5.46.126:39783
I20260928 18:00:17.776161 23293 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:17.776366 23293 heartbeater.cc:511] Master 127.5.46.126:39783 requested a full tablet report, sending...
I20260928 18:00:17.776803 22843 ts_manager.cc:194] Registered new tserver with Master: 7ef7f061891741c78e9cbfee7dc62756 (127.5.46.67:32815)
I20260928 18:00:17.777194 22843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.67:34535
I20260928 18:00:17.786835 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:17.791020 22882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.788293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.5.46.126:39783)
0928 18:00:17.788356 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.788358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.788488 (+ 130us) server_negotiation.cc:407] Connection header received
0928 18:00:17.788537 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.788538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.788548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.788583 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:17.788704 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.789181 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.789900 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.790082 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.790136 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.790764 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:17.790765 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:17.790766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:17.790770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:17.790784 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:17.790788 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:17.790831 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.790878 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.790899 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 18:00:17.792619 22843 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:17.794272 22843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:17.801497 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.5.46.65:45333)
0928 18:00:17.797062 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.797066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.797075 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:17.797113 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.797115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.797128 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.797224 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.797365 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.798572 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.800510 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.801074 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.801177 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.801274 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.801364 (+ 90us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.801402 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":130,"threads_started":1}
I20260928 18:00:17.801563 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.5.46.67:32815)
0928 18:00:17.796854 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.796860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.796873 (+ 13us) server_negotiation.cc:407] Connection header received
0928 18:00:17.796920 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.796923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.796940 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.797055 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.797331 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.798519 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.800348 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.801080 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.801165 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.801258 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.801439 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.801479 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260928 18:00:17.801486 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:45333 (local address 127.0.0.1:36804)
0928 18:00:17.796874 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.796886 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.796942 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.797223 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.797224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.797226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.797293 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.797295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.798748 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.798749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.800428 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.800430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.801182 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.801183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.801259 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.801273 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.801349 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.801404 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":48,"thread_start_us":134,"threads_started":1}
I20260928 18:00:17.801946 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:32815 (local address 127.0.0.1:57500)
0928 18:00:17.796731 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.796774 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.796841 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.797073 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.797076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.797079 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.797164 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.797166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.798741 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.798743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.800228 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.800231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.801293 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.801294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.801346 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.801349 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.801665 (+ 316us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.801712 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":42,"thread_start_us":70,"threads_started":1}
I20260928 18:00:17.802453 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:38513 (local address 127.0.0.1:45674)
0928 18:00:17.797165 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.797175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.797218 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.798867 (+ 1649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.798868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.798870 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.798945 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.798947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.800143 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.800144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.801229 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.801232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.802202 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.802203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.802268 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.802273 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.802323 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.802378 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":52,"threads_started":1}
I20260928 18:00:17.802754 22965 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b (DEFAULT_TABLE table=test-table [id=7213931a030e4e80a4b2f0152677f87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:17.802740 23227 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b (DEFAULT_TABLE table=test-table [id=7213931a030e4e80a4b2f0152677f87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:17.803123 23227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad1a3a9c4743d6976d78431ab75a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:17.803133 22965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad1a3a9c4743d6976d78431ab75a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:17.806746 23307 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Bootstrap starting.
I20260928 18:00:17.806950 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.796969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.5.46.66:38513)
0928 18:00:17.798693 (+ 1724us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.798698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.798707 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:17.798748 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.798750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.798767 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.798842 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.799213 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.800033 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.801320 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.802113 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.802185 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.806755 (+ 4570us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.806791 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.806851 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1681,"thread_start_us":116,"threads_started":1}
I20260928 18:00:17.807574 23307 tablet_bootstrap.cc:654] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:17.807689 23096 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b (DEFAULT_TABLE table=test-table [id=7213931a030e4e80a4b2f0152677f87f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:17.807967 23307 log.cc:824] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:17.808120 23096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad1a3a9c4743d6976d78431ab75a3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:17.808109 23308 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: Bootstrap starting.
I20260928 18:00:17.808817 23308 tablet_bootstrap.cc:654] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:17.809110 23307 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: No bootstrap required, opened a new log
I20260928 18:00:17.809134 23308 log.cc:824] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:17.809185 23307 ts_tablet_manager.cc:1397] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 18:00:17.809846 23308 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: No bootstrap required, opened a new log
I20260928 18:00:17.809927 23308 ts_tablet_manager.cc:1397] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:17.811517 23311 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Bootstrap starting.
I20260928 18:00:17.811908 23307 raft_consensus.cc:348] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.812186 23307 raft_consensus.cc:374] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:17.812288 23307 raft_consensus.cc:729] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ef7f061891741c78e9cbfee7dc62756, State: Initialized, Role: FOLLOWER
I20260928 18:00:17.812436 23311 tablet_bootstrap.cc:654] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:17.812487 23307 consensus_queue.cc:260] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.812152 23308 raft_consensus.cc:348] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.812579 23308 raft_consensus.cc:374] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:17.812654 23308 raft_consensus.cc:729] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a418aeea1b4a928e5feb5708ba99d6, State: Initialized, Role: FOLLOWER
I20260928 18:00:17.812850 23311 log.cc:824] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:17.812830 23308 consensus_queue.cc:260] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.812879 23307 ts_tablet_manager.cc:1428] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 18:00:17.812999 23293 heartbeater.cc:503] Master 127.5.46.126:39783 was elected leader, sending a full tablet report...
I20260928 18:00:17.813076 23308 ts_tablet_manager.cc:1428] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:17.813195 23031 heartbeater.cc:503] Master 127.5.46.126:39783 was elected leader, sending a full tablet report...
I20260928 18:00:17.813853 23311 tablet_bootstrap.cc:492] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: No bootstrap required, opened a new log
I20260928 18:00:17.813966 23311 ts_tablet_manager.cc:1397] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260928 18:00:17.816119 23311 raft_consensus.cc:348] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.816247 23311 raft_consensus.cc:374] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:17.816269 23311 raft_consensus.cc:729] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4996bc2ff2a245ea866bceffb617f029, State: Initialized, Role: FOLLOWER
I20260928 18:00:17.816385 23311 consensus_queue.cc:260] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.816643 23311 ts_tablet_manager.cc:1428] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260928 18:00:17.816708 23162 heartbeater.cc:503] Master 127.5.46.126:39783 was elected leader, sending a full tablet report...
I20260928 18:00:17.837031 23315 raft_consensus.cc:482] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:17.837174 23315 raft_consensus.cc:504] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.837517 23315 leader_election.cc:288] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8a418aeea1b4a928e5feb5708ba99d6 (127.5.46.65:45333), 7ef7f061891741c78e9cbfee7dc62756 (127.5.46.67:32815)
I20260928 18:00:17.840456 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.837747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:33001 (local address 127.5.46.65:45333)
0928 18:00:17.837791 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.837793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.837829 (+ 36us) server_negotiation.cc:407] Connection header received
0928 18:00:17.837936 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.837937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.837956 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.838021 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.838194 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.838712 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.839679 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.840095 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.840189 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.840200 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.840351 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.840389 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 18:00:17.840456 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.837686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:45333 (local address 127.5.46.66:33001)
0928 18:00:17.837765 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.837778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.837845 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.838024 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.838025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.838026 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.838108 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.838110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.838806 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.838807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.839564 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.839567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.840190 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.840191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.840265 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.840267 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.840333 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.840375 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 18:00:17.840531 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.837759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:36661 (local address 127.5.46.67:32815)
0928 18:00:17.837817 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.837819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.838002 (+ 183us) server_negotiation.cc:407] Connection header received
0928 18:00:17.838073 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.838074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.838083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.838163 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:17.838322 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.838811 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.839746 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.840169 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.840257 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.840265 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.840424 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.840460 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 18:00:17.840575 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.837688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:32815 (local address 127.5.46.66:36661)
0928 18:00:17.837921 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.837930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:17.838014 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:17.838169 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:17.838170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:17.838172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:17.838245 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.838247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.838898 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.838899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.839648 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:17.839650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.840259 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.840260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:17.840332 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.840336 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:17.840416 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 18:00:17.840453 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":63,"thread_start_us":108,"threads_started":1}
I20260928 18:00:17.841006 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad1a3a9c4743d6976d78431ab75a3b" candidate_uuid: "4996bc2ff2a245ea866bceffb617f029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" is_pre_election: true
I20260928 18:00:17.841068 23247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad1a3a9c4743d6976d78431ab75a3b" candidate_uuid: "4996bc2ff2a245ea866bceffb617f029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef7f061891741c78e9cbfee7dc62756" is_pre_election: true
I20260928 18:00:17.841197 23247 raft_consensus.cc:2454] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4996bc2ff2a245ea866bceffb617f029 in term 0.
I20260928 18:00:17.841202 22985 raft_consensus.cc:2454] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4996bc2ff2a245ea866bceffb617f029 in term 0.
I20260928 18:00:17.841462 23051 leader_election.cc:302] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4996bc2ff2a245ea866bceffb617f029, a8a418aeea1b4a928e5feb5708ba99d6; no voters:
I20260928 18:00:17.841617 23315 raft_consensus.cc:2781] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:17.841740 23315 raft_consensus.cc:482] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:17.841760 23315 raft_consensus.cc:3037] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.842648 23315 raft_consensus.cc:504] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.842796 23315 leader_election.cc:288] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [CANDIDATE]: Term 1 election: Requested vote from peers a8a418aeea1b4a928e5feb5708ba99d6 (127.5.46.65:45333), 7ef7f061891741c78e9cbfee7dc62756 (127.5.46.67:32815)
I20260928 18:00:17.842943 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad1a3a9c4743d6976d78431ab75a3b" candidate_uuid: "4996bc2ff2a245ea866bceffb617f029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8a418aeea1b4a928e5feb5708ba99d6"
I20260928 18:00:17.842965 23247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad1a3a9c4743d6976d78431ab75a3b" candidate_uuid: "4996bc2ff2a245ea866bceffb617f029" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ef7f061891741c78e9cbfee7dc62756"
I20260928 18:00:17.843035 23247 raft_consensus.cc:3037] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.843027 22985 raft_consensus.cc:3037] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:17.843971 22985 raft_consensus.cc:2454] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4996bc2ff2a245ea866bceffb617f029 in term 1.
I20260928 18:00:17.843971 23247 raft_consensus.cc:2454] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4996bc2ff2a245ea866bceffb617f029 in term 1.
I20260928 18:00:17.844210 23050 leader_election.cc:302] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4996bc2ff2a245ea866bceffb617f029, 7ef7f061891741c78e9cbfee7dc62756; no voters:
I20260928 18:00:17.844310 23315 raft_consensus.cc:2781] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:17.844491 23315 raft_consensus.cc:686] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 1 LEADER]: Becoming Leader. State: Replica: 4996bc2ff2a245ea866bceffb617f029, State: Running, Role: LEADER
I20260928 18:00:17.844601 23315 consensus_queue.cc:237] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } }
I20260928 18:00:17.845491 22842 catalog_manager.cc:5495] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4996bc2ff2a245ea866bceffb617f029 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "4996bc2ff2a245ea866bceffb617f029" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4996bc2ff2a245ea866bceffb617f029" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 38513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 } health_report { overall_health: UNKNOWN } } }
I20260928 18:00:17.859231 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.856727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.5.46.66:38513)
0928 18:00:17.856788 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:17.856790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:17.856817 (+ 27us) server_negotiation.cc:407] Connection header received
0928 18:00:17.856914 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:17.856915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:17.856938 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:17.857037 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:17.857173 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.857646 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.858399 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:17.858617 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:17.858696 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:17.858864 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:17.858975 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:17.859072 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:17.859092 (+ 20us) server_negotiation.cc:299] Negotiation successful
0928 18:00:17.859112 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 18:00:17.861387 23247 raft_consensus.cc:1261] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 1 FOLLOWER]: Refusing update from remote peer 4996bc2ff2a245ea866bceffb617f029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:17.861411 22985 raft_consensus.cc:1261] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 1 FOLLOWER]: Refusing update from remote peer 4996bc2ff2a245ea866bceffb617f029: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:17.861732 23317 consensus_queue.cc:1035] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ef7f061891741c78e9cbfee7dc62756" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 32815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:17.861871 23315 consensus_queue.cc:1035] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8a418aeea1b4a928e5feb5708ba99d6" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 45333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:17.864498 23320 mvcc.cc:205] Tried to move back new op lower bound from 7334373039557107712 to 7334373039492268032. Current Snapshot: MvccSnapshot[applied={T|T < 7334373039557107712}]
I20260928 18:00:17.865042 23322 mvcc.cc:205] Tried to move back new op lower bound from 7334373039557107712 to 7334373039492268032. Current Snapshot: MvccSnapshot[applied={T|T < 7334373039557107712}]
I20260928 18:00:17.865679 23321 mvcc.cc:205] Tried to move back new op lower bound from 7334373039557107712 to 7334373039492268032. Current Snapshot: MvccSnapshot[applied={T|T < 7334373039557107712}]
I20260928 18:00:17.867246 22842 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:17.867352 22842 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:17.868134 22842 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0f70d1e0c0042a2840c8315e717b740: Sending DeleteTablet for 3 replicas of tablet a2ad1a3a9c4743d6976d78431ab75a3b
I20260928 18:00:17.868484 23227 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='slave'} at 127.0.0.1:57500
I20260928 18:00:17.868530 23096 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='slave'} at 127.0.0.1:45674
I20260928 18:00:17.868559 22965 tablet_service.cc:1514] Processing DeleteTablet for tablet a2ad1a3a9c4743d6976d78431ab75a3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:17 UTC) from {username='slave'} at 127.0.0.1:36804
I20260928 18:00:17.868774 23332 tablet_replica.cc:330] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6: stopping tablet replica
I20260928 18:00:17.868896 23330 tablet_replica.cc:330] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: stopping tablet replica
I20260928 18:00:17.869004 23330 raft_consensus.cc:2229] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:17.869104 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22903
I20260928 18:00:17.869231 23332 raft_consensus.cc:2229] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:17.869340 23332 raft_consensus.cc:2258] T a2ad1a3a9c4743d6976d78431ab75a3b P a8a418aeea1b4a928e5feb5708ba99d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:17.869472 23330 raft_consensus.cc:2258] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:17.870018 23330 ts_tablet_manager.cc:1905] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:17.871321 23331 tablet_replica.cc:330] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: stopping tablet replica
I20260928 18:00:17.871459 23331 raft_consensus.cc:2229] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:17.871603 23331 raft_consensus.cc:2258] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:17.871678 23330 ts_tablet_manager.cc:1918] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:17.871789 23330 log.cc:1196] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/a2ad1a3a9c4743d6976d78431ab75a3b
I20260928 18:00:17.872164 23330 ts_tablet_manager.cc:1939] T a2ad1a3a9c4743d6976d78431ab75a3b P 7ef7f061891741c78e9cbfee7dc62756: Deleting consensus metadata
I20260928 18:00:17.872192 23331 ts_tablet_manager.cc:1905] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:17.872686 22828 catalog_manager.cc:4841] TS 7ef7f061891741c78e9cbfee7dc62756 (127.5.46.67:32815): tablet a2ad1a3a9c4743d6976d78431ab75a3b (table test-table [id=7213931a030e4e80a4b2f0152677f87f]) successfully deleted
I20260928 18:00:17.874137 23331 ts_tablet_manager.cc:1918] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:17.874228 23331 log.cc:1196] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/a2ad1a3a9c4743d6976d78431ab75a3b
I20260928 18:00:17.874492 23331 ts_tablet_manager.cc:1939] T a2ad1a3a9c4743d6976d78431ab75a3b P 4996bc2ff2a245ea866bceffb617f029: Deleting consensus metadata
I20260928 18:00:17.874913 22829 catalog_manager.cc:4841] TS 4996bc2ff2a245ea866bceffb617f029 (127.5.46.66:38513): tablet a2ad1a3a9c4743d6976d78431ab75a3b (table test-table [id=7213931a030e4e80a4b2f0152677f87f]) successfully deleted
W20260928 18:00:17.877205 22829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:45333 (error 108)
I20260928 18:00:17.877393 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23034
I20260928 18:00:17.878397 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:17.878217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:45333 (local address 127.0.0.1:36814)
0928 18:00:17.878296 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:17.878359 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:45333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 18:00:17.878566 22829 catalog_manager.cc:4584] TS a8a418aeea1b4a928e5feb5708ba99d6 (127.5.46.65:45333): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a2ad1a3a9c4743d6976d78431ab75a3b: Network error: Client connection negotiation failed: client connection to 127.5.46.65:45333: connect: Connection refused (error 111)
I20260928 18:00:17.881888 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23165
I20260928 18:00:17.886377 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 22812
2026-09-28T18:00:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (756 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:17 dist-test-slave-z2t3 krb5kdc[23336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:17 dist-test-slave-z2t3 krb5kdc[23336](info): set up 2 sockets
Sep 28 18:00:17 dist-test-slave-z2t3 krb5kdc[23336](info): commencing operation
krb5kdc: starting...
W20260928 18:00:18.931803 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:19.059466 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:32931
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:32931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:19.142210 23349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:19.142460 23349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:19.142486 23349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:19.143750 23349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:19.143806 23349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:19.143821 23349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:19.143834 23349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:19.143903 23349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:19.145509 23349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:32931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:32931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:19.145853 23349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:19.146123 23349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:19.151253 23355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.151258 23354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.151253 23357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:19.151611 23349 server_base.cc:1034] running on GCE node
I20260928 18:00:19.151948 23349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:19.152288 23349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:19.153461 23349 hybrid_clock.cc:648] HybridClock initialized: now 1790618419153446 us; error 31 us; skew 500 ppm
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:19.156306 23349 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:19.156818 23349 webserver.cc:460] Webserver started at http://127.5.46.126:45249/ using document root <none> and password file <none>
I20260928 18:00:19.157073 23349 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:19.157142 23349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:19.157295 23349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:19.158445 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "667965c3d8ca475a8e40e5a166e3a6d2"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "9fe70041cd5f2721d2b3727d36591ac6"
server_key_iv: "c90272b75ecb78f7e2900d43ba3e712e"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.158818 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "667965c3d8ca475a8e40e5a166e3a6d2"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "9fe70041cd5f2721d2b3727d36591ac6"
server_key_iv: "c90272b75ecb78f7e2900d43ba3e712e"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.160259 23349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:19.161118 23364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.161345 23349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:19.161437 23349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "667965c3d8ca475a8e40e5a166e3a6d2"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "9fe70041cd5f2721d2b3727d36591ac6"
server_key_iv: "c90272b75ecb78f7e2900d43ba3e712e"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.161494 23349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:19.176186 23349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:19.178639 23349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:19.178759 23349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:19.182834 23349 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:32931
I20260928 18:00:19.182906 23416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:32931 every 8 connection(s)
I20260928 18:00:19.183281 23349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:19.184322 23417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:19.184902 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23349
I20260928 18:00:19.185024 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:19.185225 5305 external_mini_cluster.cc:1428] Setting key b5cd2a6be7750d0bf89958571c7330ec
I20260928 18:00:19.187559 23417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: Bootstrap starting.
I20260928 18:00:19.188258 23417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:19.188551 23417 log.cc:824] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:19.189353 23417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: No bootstrap required, opened a new log
I20260928 18:00:19.191772 23417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } }
I20260928 18:00:19.192013 23417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:19.192050 23417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 667965c3d8ca475a8e40e5a166e3a6d2, State: Initialized, Role: FOLLOWER
I20260928 18:00:19.192236 23417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } }
I20260928 18:00:19.192340 23417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:19.192376 23417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:19.192435 23417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 0 FOLLOWER]: Advancing to term 1
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:19.193127 23417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } }
I20260928 18:00:19.193255 23417 leader_election.cc:302] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 667965c3d8ca475a8e40e5a166e3a6d2; no voters:
I20260928 18:00:19.193493 23417 leader_election.cc:288] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:19.193526 23422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:19.193663 23422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [term 1 LEADER]: Becoming Leader. State: Replica: 667965c3d8ca475a8e40e5a166e3a6d2, State: Running, Role: LEADER
I20260928 18:00:19.193784 23417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:19.193815 23422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } }
I20260928 18:00:19.194185 23424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } } }
I20260928 18:00:19.194363 23424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:19.194172 23422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 667965c3d8ca475a8e40e5a166e3a6d2. Latest consensus state: current_term: 1 leader_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "667965c3d8ca475a8e40e5a166e3a6d2" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 32931 } } }
I20260928 18:00:19.194880 23422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:19.195123 23431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:19.195676 23431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:19.197638 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.185810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.5.46.126:32931)
0928 18:00:19.185981 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.185991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.186027 (+ 36us) server_negotiation.cc:407] Connection header received
0928 18:00:19.186431 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.186437 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.186490 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.186611 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:19.187452 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.188598 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.189877 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.190358 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.190451 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.193435 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:19.193445 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:19.193445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:19.193459 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:19.195707 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.196243 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.196246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.196246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.196360 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.196589 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.196590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.196590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.196937 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:19.197102 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.197260 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.197427 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260928 18:00:19.197947 23431 catalog_manager.cc:1347] Generated new cluster ID: cfa6135c507c4f21a82d03c729dc7225
I20260928 18:00:19.198001 23431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:19.208374 23431 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:19.208935 23431 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:19.214247 23431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: Generated new TSK 0
I20260928 18:00:19.214461 23431 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 18:00:19.247591 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:19.332029 23443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:19.332309 23443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:19.332360 23443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:19.333648 23443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:19.333729 23443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:19.333779 23443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:19.335415 23443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:19.335719 23443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:19.336045 23443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:19.341995 23448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.341998 23449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.341998 23451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:19.342065 23443 server_base.cc:1034] running on GCE node
I20260928 18:00:19.342494 23443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:19.342756 23443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:19.343914 23443 hybrid_clock.cc:648] HybridClock initialized: now 1790618419343900 us; error 30 us; skew 500 ppm
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:19.346474 23443 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 18:00:19.347019 23443 webserver.cc:460] Webserver started at http://127.5.46.65:45423/ using document root <none> and password file <none>
I20260928 18:00:19.347263 23443 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:19.347316 23443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:19.347491 23443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:19.348664 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "6979a3bf0fda4e14936c5d64417edc8c"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e51ed9e504e4b0b3db260907efb5b480"
server_key_iv: "f6291f52531f358bf78309e02a65493b"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.348984 23443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "6979a3bf0fda4e14936c5d64417edc8c"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e51ed9e504e4b0b3db260907efb5b480"
server_key_iv: "f6291f52531f358bf78309e02a65493b"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.350359 23443 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 18:00:19.351300 23458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.351585 23443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:19.351712 23443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "6979a3bf0fda4e14936c5d64417edc8c"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e51ed9e504e4b0b3db260907efb5b480"
server_key_iv: "f6291f52531f358bf78309e02a65493b"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.351799 23443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:19.363313 23443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:19.366039 23443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:19.366214 23443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:19.366503 23443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:19.366866 23443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:19.366904 23443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.366930 23443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:19.366981 23443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.373507 23443 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:40013
I20260928 18:00:19.373566 23571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:40013 every 8 connection(s)
I20260928 18:00:19.373924 23443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:19.383610 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23443
I20260928 18:00:19.383764 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:19.383955 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.374697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:39957 (local address 127.5.46.126:32931)
0928 18:00:19.374795 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.374799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.375012 (+ 213us) server_negotiation.cc:407] Connection header received
0928 18:00:19.375538 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.375539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.375551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.375613 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:19.376693 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.377235 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.378339 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.378560 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.378627 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.380894 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:19.380896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:19.380896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:19.380903 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:19.382439 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.382881 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.382882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.382883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.382950 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.383220 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.383220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.383220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.383333 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:19.383456 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.383706 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.383831 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 18:00:19.384035 5305 external_mini_cluster.cc:1428] Setting key cf34f3cf2ece9a99f10c232dc59f9eaa
I20260928 18:00:19.384311 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.374663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:32931 (local address 127.5.46.65:39957)
0928 18:00:19.374855 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.374896 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.375412 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.375668 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.375671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.376356 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:19.376560 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.376565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.377328 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.377328 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.378228 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.378230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.378650 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.378651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.378776 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.379292 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:19.379300 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:19.380803 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:19.382522 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.382529 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.382531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.382780 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.383046 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.383047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.383047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.383159 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.383442 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:19.383447 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:19.383576 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.383665 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.384054 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":125,"threads_started":1}
I20260928 18:00:19.384838 23572 heartbeater.cc:347] Connected to a master server at 127.5.46.126:32931
I20260928 18:00:19.384959 23572 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:19.385169 23572 heartbeater.cc:511] Master 127.5.46.126:32931 requested a full tablet report, sending...
I20260928 18:00:19.385702 23381 ts_manager.cc:194] Registered new tserver with Master: 6979a3bf0fda4e14936c5d64417edc8c (127.5.46.65:40013)
I20260928 18:00:19.386569 23381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:39957
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 18:00:19.411629 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:19.494705 23577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:19.494953 23577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:19.494992 23577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:19.496331 23577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:19.496381 23577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:19.496410 23577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:19.497972 23577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:19.498257 23577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:19.498517 23577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:19.503779 23585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.503798 23582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.503970 23583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:19.504009 23577 server_base.cc:1034] running on GCE node
I20260928 18:00:19.504314 23577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:19.504531 23577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:19.505722 23577 hybrid_clock.cc:648] HybridClock initialized: now 1790618419505702 us; error 29 us; skew 500 ppm
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:19.508538 23577 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 18:00:19.508989 23577 webserver.cc:460] Webserver started at http://127.5.46.66:43597/ using document root <none> and password file <none>
I20260928 18:00:19.509238 23577 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:19.509305 23577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:19.509446 23577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:19.510597 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "a047a8e0cd554ae78f037e2d144a0ba8"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e94b35c30b48a5d0dbb1ce75a6c2fb1e"
server_key_iv: "240431ab1ccc8e811115419cdad5d1bb"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.510963 23577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "a047a8e0cd554ae78f037e2d144a0ba8"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e94b35c30b48a5d0dbb1ce75a6c2fb1e"
server_key_iv: "240431ab1ccc8e811115419cdad5d1bb"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.512425 23577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:19.513250 23592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.513440 23577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:19.513542 23577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "a047a8e0cd554ae78f037e2d144a0ba8"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "e94b35c30b48a5d0dbb1ce75a6c2fb1e"
server_key_iv: "240431ab1ccc8e811115419cdad5d1bb"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.513616 23577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:19.522573 23577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:19.524701 23577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:19.524866 23577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:19.525106 23577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:19.525426 23577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:19.525480 23577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.525522 23577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:19.525555 23577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.531509 23577 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33599
I20260928 18:00:19.531615 23705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33599 every 8 connection(s)
I20260928 18:00:19.531958 23577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:19.536818 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23577
I20260928 18:00:19.536960 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:19.537245 5305 external_mini_cluster.cc:1428] Setting key c3611fe921628ffaf19be45f8ce8d134
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:19.542727 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.532788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:41169 (local address 127.5.46.126:32931)
0928 18:00:19.532865 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.532869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.533052 (+ 183us) server_negotiation.cc:407] Connection header received
0928 18:00:19.533524 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.533524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.533535 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.533607 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:19.534532 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.535108 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.536227 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.536443 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.536504 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.539497 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:19.539499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:19.539500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:19.539510 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:19.541096 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.541643 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.541644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.541645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.541705 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.542008 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.542009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.542009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.542136 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:19.542191 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.542568 (+ 377us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.542644 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 18:00:19.543052 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.532750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:32931 (local address 127.5.46.66:41169)
0928 18:00:19.532921 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.532963 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.533440 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.533662 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.533665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.534241 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:19.534435 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.534440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.535218 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.535219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.536145 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.536148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.536697 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.536698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.536863 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.537578 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:19.537588 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:19.539394 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:19.541206 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.541214 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.541217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.541556 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.541777 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.541778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.541779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.541932 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.542213 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:19.542220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:19.542416 (+ 196us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.542542 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.542743 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260928 18:00:19.543768 23706 heartbeater.cc:347] Connected to a master server at 127.5.46.126:32931
I20260928 18:00:19.543912 23706 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:19.544126 23706 heartbeater.cc:511] Master 127.5.46.126:32931 requested a full tablet report, sending...
I20260928 18:00:19.544589 23381 ts_manager.cc:194] Registered new tserver with Master: a047a8e0cd554ae78f037e2d144a0ba8 (127.5.46.66:33599)
I20260928 18:00:19.545135 23381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:41169
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 18:00:19.567735 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:19.650300 23711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:19.650593 23711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:19.650651 23711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:19.651942 23711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:19.652033 23711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:19.652079 23711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:19.653662 23711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:32931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:19.653963 23711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:19.654230 23711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:19.659755 23716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:19.659888 23711 server_base.cc:1034] running on GCE node
W20260928 18:00:19.659709 23719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:19.659711 23717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:19.660190 23711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:19.660471 23711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:19.661666 23711 hybrid_clock.cc:648] HybridClock initialized: now 1790618419661664 us; error 72 us; skew 500 ppm
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:19.664515 23711 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 18:00:19.665032 23711 webserver.cc:460] Webserver started at http://127.5.46.67:40559/ using document root <none> and password file <none>
I20260928 18:00:19.665230 23711 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:19.665274 23711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:19.665411 23711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:19.666481 23711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "776ef550c30d46138bf6c04836934983"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "094207794200389d0c639a2f3ec04f8d"
server_key_iv: "eef9a060a017df0ccc1d65d8c9e25d25"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.666810 23711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "776ef550c30d46138bf6c04836934983"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "094207794200389d0c639a2f3ec04f8d"
server_key_iv: "eef9a060a017df0ccc1d65d8c9e25d25"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.668221 23711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 18:00:19.669028 23726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.669225 23711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:19.669322 23711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "776ef550c30d46138bf6c04836934983"
format_stamp: "Formatted at 2026-09-28 18:00:19 on dist-test-slave-z2t3"
server_key: "094207794200389d0c639a2f3ec04f8d"
server_key_iv: "eef9a060a017df0ccc1d65d8c9e25d25"
server_key_version: "encryptionkey@0"
I20260928 18:00:19.669400 23711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:19.689076 23711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:19.691604 23711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:19.691808 23711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:19.692101 23711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:19.692437 23711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:19.692493 23711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.692536 23711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:19.692564 23711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:19.699082 23711 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:37985
I20260928 18:00:19.699126 23839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:37985 every 8 connection(s)
I20260928 18:00:19.699452 23711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:19.703222 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23711
I20260928 18:00:19.703366 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:19.703593 5305 external_mini_cluster.cc:1428] Setting key 23682d53682a12b72649b00514ea65a7
Sep 28 18:00:19 dist-test-slave-z2t3 krb5kdc[23336](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618419, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:19.709229 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.700207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:44779 (local address 127.5.46.126:32931)
0928 18:00:19.700289 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.700292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.700472 (+ 180us) server_negotiation.cc:407] Connection header received
0928 18:00:19.700963 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.700965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.700976 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.701045 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:19.701995 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.702526 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.703612 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.703956 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.704064 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.706407 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:19.706409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:19.706409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:19.706417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:19.707810 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.708300 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.708301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.708301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.708358 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.708650 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.708650 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.708651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.708758 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:19.708809 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.709080 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.709146 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 18:00:19.709506 23842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.700167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:32931 (local address 127.5.46.67:44779)
0928 18:00:19.700352 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.700391 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.700848 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.701112 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.701116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.701695 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:19.701892 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.701896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.702612 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.702613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.703527 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.703530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.704075 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.704076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.704222 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.704748 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:19.704756 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:19.706310 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:19.707977 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.707983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.707985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.708233 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.708446 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:19.708447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:19.708448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:19.708573 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:19.708823 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:19.708829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:19.708958 (+ 129us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.709056 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.709206 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260928 18:00:19.710060 23840 heartbeater.cc:347] Connected to a master server at 127.5.46.126:32931
I20260928 18:00:19.710188 23840 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:19.710399 23840 heartbeater.cc:511] Master 127.5.46.126:32931 requested a full tablet report, sending...
I20260928 18:00:19.710811 23381 ts_manager.cc:194] Registered new tserver with Master: 776ef550c30d46138bf6c04836934983 (127.5.46.67:37985)
I20260928 18:00:19.711189 23381 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:44779
I20260928 18:00:19.715811 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:19.723534 23420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.717251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.5.46.126:32931)
0928 18:00:19.717323 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.717330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.717372 (+ 42us) server_negotiation.cc:407] Connection header received
0928 18:00:19.717429 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.717429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.717439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.717489 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:19.718150 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.718641 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.719354 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.719538 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.719595 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.721012 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:19.721013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:19.721013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:19.721020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:19.722290 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.722697 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.722697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.722698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.722750 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:19.722965 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:19.722966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:19.722966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:19.723113 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:19.723208 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.723327 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.723422 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 18:00:19.725383 23381 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:19.726974 23381 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:19.733676 23853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33599 (local address 127.0.0.1:55922)
0928 18:00:19.729844 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.729856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.729908 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.730144 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.730148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.730151 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:19.730232 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.730238 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731282 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.731283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.732546 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.732549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733355 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.733440 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.733455 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.733528 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.733585 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":108,"threads_started":1}
I20260928 18:00:19.733676 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.5.46.66:33599)
0928 18:00:19.729985 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.729990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.729998 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:19.730030 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.730032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.730044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.730132 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:19.730324 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731162 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.732633 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733269 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733356 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.733421 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.733551 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.733592 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260928 18:00:19.734098 23851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37985 (local address 127.0.0.1:48878)
0928 18:00:19.729715 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.729754 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.729821 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.730560 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.730561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.730563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:19.730612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.730613 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731839 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.731840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.732884 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.732887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733840 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.733919 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.733923 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.733977 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.734017 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260928 18:00:19.734136 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48878 (local address 127.5.46.67:37985)
0928 18:00:19.730345 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.730349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.730359 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:19.730399 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.730401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.730415 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.730485 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:19.730701 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731735 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.732976 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733664 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733745 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.733802 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.734009 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.734058 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":82,"threads_started":1}
I20260928 18:00:19.734570 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.5.46.65:40013)
0928 18:00:19.730050 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.730054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.730063 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:19.730099 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.730101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.730126 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.730212 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:19.730365 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731496 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.732782 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733496 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733569 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.733622 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.734446 (+ 824us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.734493 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260928 18:00:19.734478 23640 tablet_service.cc:1467] Processing CreateTablet for tablet bd1573a784264af28d8dbadfa241fb03 (DEFAULT_TABLE table=test-table [id=671fe86086224b4799260047c0c6c30d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:19.734848 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.729535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:40013 (local address 127.0.0.1:55316)
0928 18:00:19.729944 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.729953 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.729991 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.730211 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.730212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.730214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:19.730283 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.730285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.731609 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.731610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.732688 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.732695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.733661 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.733662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.734082 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.734086 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.734137 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.734779 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":16,"thread_start_us":93,"threads_started":1}
I20260928 18:00:19.734786 23774 tablet_service.cc:1467] Processing CreateTablet for tablet bd1573a784264af28d8dbadfa241fb03 (DEFAULT_TABLE table=test-table [id=671fe86086224b4799260047c0c6c30d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:19.734881 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1573a784264af28d8dbadfa241fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:19.735109 23774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1573a784264af28d8dbadfa241fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:19.736295 23506 tablet_service.cc:1467] Processing CreateTablet for tablet bd1573a784264af28d8dbadfa241fb03 (DEFAULT_TABLE table=test-table [id=671fe86086224b4799260047c0c6c30d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:19.736624 23506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd1573a784264af28d8dbadfa241fb03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:19.738257 23858 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Bootstrap starting.
I20260928 18:00:19.738857 23858 tablet_bootstrap.cc:654] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:19.739367 23858 log.cc:824] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:19.739706 23860 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Bootstrap starting.
I20260928 18:00:19.739871 23857 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Bootstrap starting.
I20260928 18:00:19.740245 23858 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: No bootstrap required, opened a new log
I20260928 18:00:19.740331 23858 ts_tablet_manager.cc:1397] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:19.740491 23857 tablet_bootstrap.cc:654] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:19.740491 23860 tablet_bootstrap.cc:654] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:19.741123 23857 log.cc:824] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:19.741123 23860 log.cc:824] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:19.741897 23857 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: No bootstrap required, opened a new log
I20260928 18:00:19.741977 23857 ts_tablet_manager.cc:1397] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:19.741976 23860 tablet_bootstrap.cc:492] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: No bootstrap required, opened a new log
I20260928 18:00:19.742043 23860 ts_tablet_manager.cc:1397] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:19.742331 23858 raft_consensus.cc:348] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.742539 23858 raft_consensus.cc:374] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:19.742573 23858 raft_consensus.cc:729] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776ef550c30d46138bf6c04836934983, State: Initialized, Role: FOLLOWER
I20260928 18:00:19.742720 23858 consensus_queue.cc:260] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.743026 23858 ts_tablet_manager.cc:1428] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:19.743044 23840 heartbeater.cc:503] Master 127.5.46.126:32931 was elected leader, sending a full tablet report...
I20260928 18:00:19.743938 23857 raft_consensus.cc:348] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.744014 23860 raft_consensus.cc:348] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.744139 23857 raft_consensus.cc:374] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:19.744186 23857 raft_consensus.cc:729] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a047a8e0cd554ae78f037e2d144a0ba8, State: Initialized, Role: FOLLOWER
I20260928 18:00:19.744187 23860 raft_consensus.cc:374] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:19.744233 23860 raft_consensus.cc:729] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6979a3bf0fda4e14936c5d64417edc8c, State: Initialized, Role: FOLLOWER
I20260928 18:00:19.744307 23857 consensus_queue.cc:260] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.744355 23860 consensus_queue.cc:260] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.744678 23860 ts_tablet_manager.cc:1428] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:19.744668 23706 heartbeater.cc:503] Master 127.5.46.126:32931 was elected leader, sending a full tablet report...
I20260928 18:00:19.744675 23857 ts_tablet_manager.cc:1428] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:19.744925 23572 heartbeater.cc:503] Master 127.5.46.126:32931 was elected leader, sending a full tablet report...
W20260928 18:00:19.782646 23707 tablet.cc:2369] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 18:00:19.874791 23573 tablet.cc:2369] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 18:00:19.910080 23864 raft_consensus.cc:482] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:19.910269 23864 raft_consensus.cc:504] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.910749 23864 leader_election.cc:288] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 776ef550c30d46138bf6c04836934983 (127.5.46.67:37985), 6979a3bf0fda4e14936c5d64417edc8c (127.5.46.65:40013)
I20260928 18:00:19.913993 23856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.911159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:53049 (local address 127.5.46.65:40013)
0928 18:00:19.911207 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.911209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.911220 (+ 11us) server_negotiation.cc:407] Connection header received
0928 18:00:19.911253 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.911254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.911268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.911332 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:19.911516 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.912309 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913265 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.913629 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913715 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.913735 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.913890 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.913925 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 18:00:19.913993 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.910996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:40013 (local address 127.5.46.66:53049)
0928 18:00:19.911083 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.911097 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.911169 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.911352 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.911353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.911355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:19.911434 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.911436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.912390 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.912391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.913148 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.913151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.913714 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.913803 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.913806 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.913871 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.913917 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 18:00:19.914072 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.911167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48297 (local address 127.5.46.67:37985)
0928 18:00:19.911224 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.911226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.911331 (+ 105us) server_negotiation.cc:407] Connection header received
0928 18:00:19.911366 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.911367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.911379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.911444 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:19.911719 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.912370 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913317 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.913709 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913797 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.913807 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.913951 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.913990 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 18:00:19.914310 23866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.910996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:37985 (local address 127.5.46.66:48297)
0928 18:00:19.911248 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.911258 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:19.911321 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:19.911458 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:19.911458 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:19.911460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:19.911578 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.911580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.912467 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.912468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.913219 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:19.913221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.913801 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.913802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:19.913876 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.913879 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:19.913945 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 18:00:19.913984 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":87,"thread_start_us":92,"threads_started":1}
I20260928 18:00:19.914340 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1573a784264af28d8dbadfa241fb03" candidate_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6979a3bf0fda4e14936c5d64417edc8c" is_pre_election: true
I20260928 18:00:19.914541 23526 raft_consensus.cc:2454] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a047a8e0cd554ae78f037e2d144a0ba8 in term 0.
I20260928 18:00:19.914742 23794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1573a784264af28d8dbadfa241fb03" candidate_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776ef550c30d46138bf6c04836934983" is_pre_election: true
I20260928 18:00:19.914860 23794 raft_consensus.cc:2454] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a047a8e0cd554ae78f037e2d144a0ba8 in term 0.
I20260928 18:00:19.914806 23595 leader_election.cc:302] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6979a3bf0fda4e14936c5d64417edc8c, a047a8e0cd554ae78f037e2d144a0ba8; no voters:
I20260928 18:00:19.914954 23864 raft_consensus.cc:2781] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:19.915028 23864 raft_consensus.cc:482] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:19.915107 23864 raft_consensus.cc:3037] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:19.915840 23864 raft_consensus.cc:504] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.916049 23864 leader_election.cc:288] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [CANDIDATE]: Term 1 election: Requested vote from peers 776ef550c30d46138bf6c04836934983 (127.5.46.67:37985), 6979a3bf0fda4e14936c5d64417edc8c (127.5.46.65:40013)
I20260928 18:00:19.916180 23794 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1573a784264af28d8dbadfa241fb03" candidate_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "776ef550c30d46138bf6c04836934983"
I20260928 18:00:19.916231 23526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd1573a784264af28d8dbadfa241fb03" candidate_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6979a3bf0fda4e14936c5d64417edc8c"
I20260928 18:00:19.916296 23794 raft_consensus.cc:3037] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:19.916301 23526 raft_consensus.cc:3037] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:19.916999 23526 raft_consensus.cc:2454] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a047a8e0cd554ae78f037e2d144a0ba8 in term 1.
I20260928 18:00:19.917104 23794 raft_consensus.cc:2454] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a047a8e0cd554ae78f037e2d144a0ba8 in term 1.
I20260928 18:00:19.917194 23595 leader_election.cc:302] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6979a3bf0fda4e14936c5d64417edc8c, a047a8e0cd554ae78f037e2d144a0ba8; no voters:
I20260928 18:00:19.917289 23864 raft_consensus.cc:2781] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:19.917487 23864 raft_consensus.cc:686] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 1 LEADER]: Becoming Leader. State: Replica: a047a8e0cd554ae78f037e2d144a0ba8, State: Running, Role: LEADER
I20260928 18:00:19.917623 23864 consensus_queue.cc:237] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } }
I20260928 18:00:19.918556 23381 catalog_manager.cc:5495] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a047a8e0cd554ae78f037e2d144a0ba8 (127.5.46.66). New cstate: current_term: 1 leader_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a047a8e0cd554ae78f037e2d144a0ba8" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33599 } health_report { overall_health: HEALTHY } } }
W20260928 18:00:19.950060 23841 tablet.cc:2369] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 18:00:19.962363 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.959471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55926 (local address 127.5.46.66:33599)
0928 18:00:19.959544 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:19.959546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:19.959557 (+ 11us) server_negotiation.cc:407] Connection header received
0928 18:00:19.959653 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:19.959654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:19.959679 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:19.959772 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:19.959948 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.960439 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.961257 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:19.961461 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:19.961555 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:19.961882 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:19.961977 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:19.962220 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:19.962261 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 18:00:19.962272 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 18:00:19.964318 23526 raft_consensus.cc:1261] T bd1573a784264af28d8dbadfa241fb03 P 6979a3bf0fda4e14936c5d64417edc8c [term 1 FOLLOWER]: Refusing update from remote peer a047a8e0cd554ae78f037e2d144a0ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:19.964385 23794 raft_consensus.cc:1261] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 1 FOLLOWER]: Refusing update from remote peer a047a8e0cd554ae78f037e2d144a0ba8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:19.964787 23864 consensus_queue.cc:1035] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6979a3bf0fda4e14936c5d64417edc8c" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:19.964956 23867 consensus_queue.cc:1035] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "776ef550c30d46138bf6c04836934983" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 37985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:19.968135 23871 mvcc.cc:205] Tried to move back new op lower bound from 7334373048170803200 to 7334373047983349760. Current Snapshot: MvccSnapshot[applied={T|T < 7334373048170803200}]
I20260928 18:00:19.970398 23380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:19.970510 23380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:19.971335 23380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 667965c3d8ca475a8e40e5a166e3a6d2: Sending DeleteTablet for 3 replicas of tablet bd1573a784264af28d8dbadfa241fb03
I20260928 18:00:19.971701 23774 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1573a784264af28d8dbadfa241fb03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:19 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:48878
I20260928 18:00:19.971757 23640 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1573a784264af28d8dbadfa241fb03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:19 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:55922
I20260928 18:00:19.971820 23506 tablet_service.cc:1514] Processing DeleteTablet for tablet bd1573a784264af28d8dbadfa241fb03 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:19 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:55316
I20260928 18:00:19.971969 23880 tablet_replica.cc:330] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: stopping tablet replica
I20260928 18:00:19.971982 23881 tablet_replica.cc:330] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: stopping tablet replica
I20260928 18:00:19.972065 23881 raft_consensus.cc:2229] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:19.972065 23880 raft_consensus.cc:2229] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:19.972157 23881 raft_consensus.cc:2258] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:19.972162 23880 raft_consensus.cc:2258] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:19.972553 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23443
I20260928 18:00:19.973627 23881 ts_tablet_manager.cc:1905] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:19.975718 23881 ts_tablet_manager.cc:1918] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:19.975795 23881 log.cc:1196] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/bd1573a784264af28d8dbadfa241fb03
I20260928 18:00:19.976109 23881 ts_tablet_manager.cc:1939] T bd1573a784264af28d8dbadfa241fb03 P a047a8e0cd554ae78f037e2d144a0ba8: Deleting consensus metadata
I20260928 18:00:19.976574 23365 catalog_manager.cc:4841] TS a047a8e0cd554ae78f037e2d144a0ba8 (127.5.46.66:33599): tablet bd1573a784264af28d8dbadfa241fb03 (table test-table [id=671fe86086224b4799260047c0c6c30d]) successfully deleted
I20260928 18:00:19.976778 23880 ts_tablet_manager.cc:1905] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:19.978706 23880 ts_tablet_manager.cc:1918] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:19.978785 23880 log.cc:1196] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/bd1573a784264af28d8dbadfa241fb03
I20260928 18:00:19.979043 23880 ts_tablet_manager.cc:1939] T bd1573a784264af28d8dbadfa241fb03 P 776ef550c30d46138bf6c04836934983: Deleting consensus metadata
I20260928 18:00:19.979472 23366 catalog_manager.cc:4841] TS 776ef550c30d46138bf6c04836934983 (127.5.46.67:37985): tablet bd1573a784264af28d8dbadfa241fb03 (table test-table [id=671fe86086224b4799260047c0c6c30d]) successfully deleted
W20260928 18:00:19.980340 23367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:40013 (error 108)
I20260928 18:00:19.980890 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:19.980750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:40013 (local address 127.0.0.1:55326)
0928 18:00:19.980815 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:19.980853 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 18:00:19.980958 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23577
W20260928 18:00:19.982383 23367 catalog_manager.cc:4584] TS 6979a3bf0fda4e14936c5d64417edc8c (127.5.46.65:40013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd1573a784264af28d8dbadfa241fb03: Network error: Client connection negotiation failed: client connection to 127.5.46.65:40013: connect: Connection refused (error 111)
I20260928 18:00:19.986608 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23711
I20260928 18:00:19.991395 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23349
2026-09-28T18:00:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2106 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T18:00:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:20Z Disabled control of system clock
I20260928 18:00:20.020937 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:40205
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:37589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:40205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:20.104403 23887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:20.104663 23887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:20.104687 23887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:20.106029 23887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:20.106081 23887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:20.106094 23887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:20.106106 23887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:20.106119 23887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:20.107547 23887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:40205
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:40205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:20.107836 23887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:20.108206 23887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 18:00:20.113490 23887 server_base.cc:1034] running on GCE node
W20260928 18:00:20.113456 23893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.113471 23892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.113471 23895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:20.114025 23887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:20.114300 23887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:20.115476 23887 hybrid_clock.cc:648] HybridClock initialized: now 1790618420115451 us; error 38 us; skew 500 ppm
I20260928 18:00:20.122274 23887 webserver.cc:460] Webserver started at http://127.5.46.126:42703/ using document root <none> and password file <none>
I20260928 18:00:20.122537 23887 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:20.122586 23887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:20.122687 23887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:20.123790 23887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "0ee8fa40b4377ef879dd15898082ae5e"
server_key_iv: "33e3be2f9d25285460b41211da43a085"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.124189 23887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "0ee8fa40b4377ef879dd15898082ae5e"
server_key_iv: "33e3be2f9d25285460b41211da43a085"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.125576 23887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:20.126428 23901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.126677 23887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:20.126807 23887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "0ee8fa40b4377ef879dd15898082ae5e"
server_key_iv: "33e3be2f9d25285460b41211da43a085"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.126863 23887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:20.135492 23887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:20.135931 23887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:20.136050 23887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:20.139837 23887 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:40205
I20260928 18:00:20.139957 23953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:40205 every 8 connection(s)
I20260928 18:00:20.140305 23887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:20.141294 23954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:20.143682 23954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: Bootstrap starting.
I20260928 18:00:20.144364 23954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:20.144680 23954 log.cc:824] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:20.145385 23954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: No bootstrap required, opened a new log
I20260928 18:00:20.145901 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23887
I20260928 18:00:20.145987 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:20.146184 5305 external_mini_cluster.cc:1428] Setting key 24c2d06a9e1d54d253f73fa3aaa88474
I20260928 18:00:20.147569 23954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } }
I20260928 18:00:20.147766 23954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:20.147811 23954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e616c50e5164ec09e8bd10c2dd1ab4f, State: Initialized, Role: FOLLOWER
I20260928 18:00:20.147996 23954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } }
I20260928 18:00:20.148070 23954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:20.148093 23954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:20.148159 23954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:20.148763 23954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } }
I20260928 18:00:20.148850 23954 leader_election.cc:302] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e616c50e5164ec09e8bd10c2dd1ab4f; no voters:
I20260928 18:00:20.149065 23954 leader_election.cc:288] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:20.149099 23959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:20.149207 23959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [term 1 LEADER]: Becoming Leader. State: Replica: 7e616c50e5164ec09e8bd10c2dd1ab4f, State: Running, Role: LEADER
I20260928 18:00:20.149290 23959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } }
I20260928 18:00:20.149348 23954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:20.149606 23961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } } }
I20260928 18:00:20.149844 23961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:20.149778 23960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e616c50e5164ec09e8bd10c2dd1ab4f. Latest consensus state: current_term: 1 leader_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e616c50e5164ec09e8bd10c2dd1ab4f" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 40205 } } }
I20260928 18:00:20.150097 23960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:20.150477 23968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:20.150954 23968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:20.152720 23968 catalog_manager.cc:1347] Generated new cluster ID: ad84fbbb2e3a46ec8d92c31676a70da8
I20260928 18:00:20.152796 23968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:20.153155 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.146767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40510 (local address 127.5.46.126:40205)
0928 18:00:20.146956 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.146963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.146989 (+ 26us) server_negotiation.cc:407] Connection header received
0928 18:00:20.147490 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.147497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.147565 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.147678 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:20.147969 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.149400 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.150583 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.151348 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.151481 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.152303 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:20.152307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:20.152308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:20.152537 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:20.152580 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:20.152595 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:20.152669 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.152871 (+ 202us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.152954 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260928 18:00:20.159686 23968 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:20.160285 23968 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:20.165802 23968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: Generated new TSK 0
I20260928 18:00:20.166019 23968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:20.167753 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--builtin_ntp_servers=127.5.46.84:37589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:20.251098 23978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:20.251350 23978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:20.251391 23978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:20.252772 23978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:20.252847 23978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:20.252910 23978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:20.254372 23978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:20.254659 23978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:20.254981 23978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:20.260164 23986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.260157 23984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.260157 23983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:20.260459 23978 server_base.cc:1034] running on GCE node
I20260928 18:00:20.260665 23978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:20.260967 23978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:20.262161 23978 hybrid_clock.cc:648] HybridClock initialized: now 1790618420262148 us; error 31 us; skew 500 ppm
I20260928 18:00:20.263690 23978 webserver.cc:460] Webserver started at http://127.5.46.65:43777/ using document root <none> and password file <none>
I20260928 18:00:20.264003 23978 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:20.264065 23978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:20.264207 23978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:20.265586 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "49c1389eb3de4067aa010c52f0048844"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "ca0fb402163c640393b180c445b31739"
server_key_iv: "60b10d8a515f5c478869ba00240c86db"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.265980 23978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "49c1389eb3de4067aa010c52f0048844"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "ca0fb402163c640393b180c445b31739"
server_key_iv: "60b10d8a515f5c478869ba00240c86db"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.267628 23978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:20.268648 23992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.268922 23978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:20.269028 23978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "49c1389eb3de4067aa010c52f0048844"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "ca0fb402163c640393b180c445b31739"
server_key_iv: "60b10d8a515f5c478869ba00240c86db"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.269104 23978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:20.294317 23978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:20.294612 23978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:20.294721 23978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:20.294924 23978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:20.295287 23978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:20.295322 23978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.295379 23978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:20.295404 23978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.301275 23978 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:41797
I20260928 18:00:20.301364 24105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:41797 every 8 connection(s)
I20260928 18:00:20.301720 23978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 18:00:20.303254 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 23978
I20260928 18:00:20.303393 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:20.303634 5305 external_mini_cluster.cc:1428] Setting key e0259e283c164e29b99baaee6f993d13
I20260928 18:00:20.304924 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--builtin_ntp_servers=127.5.46.84:37589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 18:00:20.308252 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.302530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:58037 (local address 127.5.46.126:40205)
0928 18:00:20.302589 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.302591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.302912 (+ 321us) server_negotiation.cc:407] Connection header received
0928 18:00:20.303507 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.303508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.303527 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.303588 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:20.304664 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.305448 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.306609 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.306849 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.306936 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.307950 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:20.307951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:20.307952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:20.307957 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:20.307975 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:20.307980 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:20.308050 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.308155 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.308182 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 18:00:20.308552 24108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.302522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40205 (local address 127.5.46.65:58037)
0928 18:00:20.302762 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.302812 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.303355 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.303665 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.303669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.303680 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:20.304479 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.304486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.305575 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.305576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.306397 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.306399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.306946 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.306947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.307095 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.307688 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:20.307689 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:20.307736 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:20.307768 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:20.308060 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:20.308067 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:20.308070 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.308147 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.308269 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260928 18:00:20.309226 24106 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40205
I20260928 18:00:20.309373 24106 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:20.309599 24106 heartbeater.cc:511] Master 127.5.46.126:40205 requested a full tablet report, sending...
I20260928 18:00:20.310230 23918 ts_manager.cc:194] Registered new tserver with Master: 49c1389eb3de4067aa010c52f0048844 (127.5.46.65:41797)
I20260928 18:00:20.310925 23918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.65:58037
W20260928 18:00:20.389142 24109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:20.389418 24109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:20.389458 24109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:20.390766 24109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:20.390834 24109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:20.390877 24109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:20.392315 24109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:20.392611 24109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:20.392932 24109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:20.398085 24117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.398109 24115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.398118 24114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:20.398489 24109 server_base.cc:1034] running on GCE node
I20260928 18:00:20.398723 24109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:20.398940 24109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:20.400079 24109 hybrid_clock.cc:648] HybridClock initialized: now 1790618420400057 us; error 35 us; skew 500 ppm
I20260928 18:00:20.401391 24109 webserver.cc:460] Webserver started at http://127.5.46.66:39535/ using document root <none> and password file <none>
I20260928 18:00:20.401612 24109 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:20.401654 24109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:20.401746 24109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:20.402837 24109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "b614ef46fac741c09679ef42b45965ab"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "4c1da3bdcfc5581cbb4f48806a3aad8f"
server_key_iv: "a972a20eac8fdef8f3decd8c147906d5"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.403142 24109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "b614ef46fac741c09679ef42b45965ab"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "4c1da3bdcfc5581cbb4f48806a3aad8f"
server_key_iv: "a972a20eac8fdef8f3decd8c147906d5"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.404577 24109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:20.405345 24123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.405543 24109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:20.405659 24109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "b614ef46fac741c09679ef42b45965ab"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "4c1da3bdcfc5581cbb4f48806a3aad8f"
server_key_iv: "a972a20eac8fdef8f3decd8c147906d5"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.405742 24109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:20.417104 24109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:20.417464 24109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:20.417625 24109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:20.417860 24109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:20.418202 24109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:20.418257 24109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.418300 24109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:20.418327 24109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.424304 24109 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:45529
I20260928 18:00:20.424381 24236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:45529 every 8 connection(s)
I20260928 18:00:20.424770 24109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:20.430758 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.425605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:38813 (local address 127.5.46.126:40205)
0928 18:00:20.425681 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.425683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.425970 (+ 287us) server_negotiation.cc:407] Connection header received
0928 18:00:20.426458 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.426459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.426470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.426550 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:20.427336 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.427933 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.429026 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.429261 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.429343 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.430419 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:20.430420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:20.430420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:20.430425 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:20.430440 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:20.430444 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:20.430494 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.430574 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.430598 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 18:00:20.430917 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24109
I20260928 18:00:20.430974 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.425594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40205 (local address 127.5.46.66:38813)
0928 18:00:20.425806 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.425849 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.426347 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.426667 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.426671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.426680 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:20.427225 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.427230 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.428044 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.428045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.428926 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.428928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.429360 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.429361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.429488 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.430279 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:20.430279 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:20.430309 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:20.430334 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:20.430503 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:20.430507 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:20.430510 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.430563 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.430687 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260928 18:00:20.431018 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:20.431234 5305 external_mini_cluster.cc:1428] Setting key 66378997e5ef7236916562aa401087a5
I20260928 18:00:20.431808 24237 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40205
I20260928 18:00:20.431941 24237 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:20.432121 24237 heartbeater.cc:511] Master 127.5.46.126:40205 requested a full tablet report, sending...
I20260928 18:00:20.432349 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--builtin_ntp_servers=127.5.46.84:37589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 18:00:20.432518 23918 ts_manager.cc:194] Registered new tserver with Master: b614ef46fac741c09679ef42b45965ab (127.5.46.66:45529)
I20260928 18:00:20.432871 23918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.66:38813
W20260928 18:00:20.518498 24240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:20.518736 24240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:20.518760 24240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:20.520150 24240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:20.520200 24240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:20.520231 24240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:20.521667 24240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:37589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:40205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:20.521960 24240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:20.522295 24240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:20.528012 24245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.528002 24246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:20.527997 24248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:20.528321 24240 server_base.cc:1034] running on GCE node
I20260928 18:00:20.528561 24240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:20.528849 24240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:20.530014 24240 hybrid_clock.cc:648] HybridClock initialized: now 1790618420530009 us; error 40 us; skew 500 ppm
I20260928 18:00:20.531509 24240 webserver.cc:460] Webserver started at http://127.5.46.67:37855/ using document root <none> and password file <none>
I20260928 18:00:20.531791 24240 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:20.531900 24240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:20.532063 24240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:20.533203 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "38d483625e4940c686c5a1d66e1630e6"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "2678d453e7abc1dfb5e207d108df599c"
server_key_iv: "41760ae9d9d406d4615c13fafbbceec9"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.533592 24240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "38d483625e4940c686c5a1d66e1630e6"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "2678d453e7abc1dfb5e207d108df599c"
server_key_iv: "41760ae9d9d406d4615c13fafbbceec9"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.534971 24240 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:20.535737 24254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.535960 24240 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:20.536055 24240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "38d483625e4940c686c5a1d66e1630e6"
format_stamp: "Formatted at 2026-09-28 18:00:20 on dist-test-slave-z2t3"
server_key: "2678d453e7abc1dfb5e207d108df599c"
server_key_iv: "41760ae9d9d406d4615c13fafbbceec9"
server_key_version: "encryptionkey@0"
I20260928 18:00:20.536145 24240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:20.554409 24240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:20.554787 24240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:20.554958 24240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:20.555224 24240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:20.555609 24240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:20.555665 24240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.555703 24240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:20.555733 24240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:20.561947 24240 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:34921
I20260928 18:00:20.562039 24367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:34921 every 8 connection(s)
I20260928 18:00:20.562415 24240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:20.567988 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24240
I20260928 18:00:20.568128 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:20.568214 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.563281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:49643 (local address 127.5.46.126:40205)
0928 18:00:20.563356 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.563359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.563618 (+ 259us) server_negotiation.cc:407] Connection header received
0928 18:00:20.564251 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.564252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.564263 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.564340 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:20.565057 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.565542 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.566635 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.566860 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.566927 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.567828 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:20.567829 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:20.567830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:20.567835 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:20.567883 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:20.567888 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:20.567971 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.568069 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.568102 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 18:00:20.568362 5305 external_mini_cluster.cc:1428] Setting key 0c52fe79cd81ebf59fc82dfb22f573b6
I20260928 18:00:20.568508 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.563243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:40205 (local address 127.5.46.67:49643)
0928 18:00:20.563478 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.563525 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.564154 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.564416 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.564420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.564428 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:20.564980 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.564985 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.565622 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.565622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.566538 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.566541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.566954 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.566955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.567079 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.567665 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:20.567666 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:20.567699 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:20.567727 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:20.567981 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:20.567986 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:20.567988 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.568053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.568199 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
I20260928 18:00:20.569048 24368 heartbeater.cc:347] Connected to a master server at 127.5.46.126:40205
I20260928 18:00:20.569175 24368 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:20.569396 24368 heartbeater.cc:511] Master 127.5.46.126:40205 requested a full tablet report, sending...
I20260928 18:00:20.569801 23918 ts_manager.cc:194] Registered new tserver with Master: 38d483625e4940c686c5a1d66e1630e6 (127.5.46.67:34921)
I20260928 18:00:20.570180 23918 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.67:49643
I20260928 18:00:20.581014 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:20.586036 23958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.582655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40516 (local address 127.5.46.126:40205)
0928 18:00:20.582785 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.582787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.582854 (+ 67us) server_negotiation.cc:407] Connection header received
0928 18:00:20.582951 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.582952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.582964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.583036 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:20.583246 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.583754 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.584608 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.584818 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.584901 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.585713 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:20.585714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:20.585714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:20.585720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:20.585743 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:20.585746 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:20.585850 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.585924 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.585936 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 18:00:20.587947 23918 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:20.589603 23918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:20.596263 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57686 (local address 127.5.46.66:45529)
0928 18:00:20.592433 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.592441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.592455 (+ 14us) server_negotiation.cc:407] Connection header received
0928 18:00:20.592487 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.592489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.592501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.592589 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:20.592834 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.593695 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.594935 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.595664 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.595730 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.595820 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.596131 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.596181 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
I20260928 18:00:20.596251 24376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:45529 (local address 127.0.0.1:57686)
0928 18:00:20.592225 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.592267 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.592321 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.592594 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.592597 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.592599 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:20.592746 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.592749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.593781 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.593782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.594834 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.594837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.595933 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.595934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.596010 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.596020 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.596106 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.596162 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260928 18:00:20.596478 24378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41797 (local address 127.0.0.1:58890)
0928 18:00:20.592705 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.592715 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.592773 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.593042 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.593043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.593045 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:20.593094 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.593096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.594439 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.594440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.595324 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.595327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.596244 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.596245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.596303 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.596308 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.596360 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.596401 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":98,"threads_started":1}
I20260928 18:00:20.596819 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.5.46.65:41797)
0928 18:00:20.592600 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.592605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.592832 (+ 227us) server_negotiation.cc:407] Connection header received
0928 18:00:20.592879 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.592881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.592899 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.592973 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:20.593173 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.594329 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.595415 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.596056 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.596124 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.596208 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.596667 (+ 459us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.596724 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
I20260928 18:00:20.596904 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:34921 (local address 127.0.0.1:52688)
0928 18:00:20.592225 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.592268 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.592326 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.593037 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.593038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.593040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:20.593079 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.593081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.594567 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.594569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.595644 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.595647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.596653 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.596654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.596718 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.596722 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.596795 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.596839 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"spinlock_wait_cycles":384,"thread_start_us":127,"threads_started":1}
I20260928 18:00:20.597396 24171 tablet_service.cc:1467] Processing CreateTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 (DEFAULT_TABLE table=test-table [id=c3e99b62df5349bca0f9f200f2de3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:20.597425 24040 tablet_service.cc:1467] Processing CreateTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 (DEFAULT_TABLE table=test-table [id=c3e99b62df5349bca0f9f200f2de3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:20.597713 24040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cda965fcc5a43338a36fadbf74bf853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:20.597744 24171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cda965fcc5a43338a36fadbf74bf853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:20.600088 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.592686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.5.46.67:34921)
0928 18:00:20.592887 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.592892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.592900 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:20.592936 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.592938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.592953 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.593021 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:20.593172 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.594315 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.595745 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.596543 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.596649 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.596737 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.596807 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.596850 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":97,"threads_started":1}
I20260928 18:00:20.600435 24382 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Bootstrap starting.
I20260928 18:00:20.600617 24383 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Bootstrap starting.
I20260928 18:00:20.601176 24383 tablet_bootstrap.cc:654] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:20.601183 24382 tablet_bootstrap.cc:654] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:20.601260 24302 tablet_service.cc:1467] Processing CreateTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 (DEFAULT_TABLE table=test-table [id=c3e99b62df5349bca0f9f200f2de3edd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:20.601526 24382 log.cc:824] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:20.601528 24383 log.cc:824] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:20.601629 24302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cda965fcc5a43338a36fadbf74bf853. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:20.602288 24383 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: No bootstrap required, opened a new log
I20260928 18:00:20.602346 24382 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: No bootstrap required, opened a new log
I20260928 18:00:20.602386 24383 ts_tablet_manager.cc:1397] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:20.602433 24382 ts_tablet_manager.cc:1397] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:20.604568 24383 raft_consensus.cc:348] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.604740 24383 raft_consensus.cc:374] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:20.604779 24383 raft_consensus.cc:729] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b614ef46fac741c09679ef42b45965ab, State: Initialized, Role: FOLLOWER
I20260928 18:00:20.604897 24383 consensus_queue.cc:260] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.605209 24383 ts_tablet_manager.cc:1428] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:20.605242 24237 heartbeater.cc:503] Master 127.5.46.126:40205 was elected leader, sending a full tablet report...
I20260928 18:00:20.605228 24382 raft_consensus.cc:348] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.605392 24382 raft_consensus.cc:374] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:20.605436 24382 raft_consensus.cc:729] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c1389eb3de4067aa010c52f0048844, State: Initialized, Role: FOLLOWER
I20260928 18:00:20.605554 24382 consensus_queue.cc:260] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.605732 24387 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Bootstrap starting.
I20260928 18:00:20.605830 24382 ts_tablet_manager.cc:1428] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 18:00:20.605968 24106 heartbeater.cc:503] Master 127.5.46.126:40205 was elected leader, sending a full tablet report...
I20260928 18:00:20.606410 24387 tablet_bootstrap.cc:654] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:20.606741 24387 log.cc:824] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:20.607456 24387 tablet_bootstrap.cc:492] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: No bootstrap required, opened a new log
I20260928 18:00:20.607511 24387 ts_tablet_manager.cc:1397] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:20.609620 24387 raft_consensus.cc:348] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.609799 24387 raft_consensus.cc:374] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:20.609838 24387 raft_consensus.cc:729] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38d483625e4940c686c5a1d66e1630e6, State: Initialized, Role: FOLLOWER
I20260928 18:00:20.609925 24387 consensus_queue.cc:260] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.610131 24387 ts_tablet_manager.cc:1428] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 18:00:20.610219 24368 heartbeater.cc:503] Master 127.5.46.126:40205 was elected leader, sending a full tablet report...
W20260928 18:00:20.675396 24238 tablet.cc:2369] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 18:00:20.719786 24386 raft_consensus.cc:482] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:20.719954 24386 raft_consensus.cc:504] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.720325 24386 leader_election.cc:288] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38d483625e4940c686c5a1d66e1630e6 (127.5.46.67:34921), 49c1389eb3de4067aa010c52f0048844 (127.5.46.65:41797)
I20260928 18:00:20.723232 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.720522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41797 (local address 127.5.46.66:54023)
0928 18:00:20.720585 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.720597 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.720654 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.720832 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.720833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.720835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:20.720907 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.720909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.721568 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.721569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.722388 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.722391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.722950 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.722951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.723021 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.723025 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.723103 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.723150 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 18:00:20.723232 24380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.720633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:54023 (local address 127.5.46.65:41797)
0928 18:00:20.720682 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.720684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.720694 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:20.720724 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.720725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.720743 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.720822 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:20.720996 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.721468 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.722495 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.722851 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.722948 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.722959 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.723110 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.723154 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 18:00:20.723402 24381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.720607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:45681 (local address 127.5.46.67:34921)
0928 18:00:20.720662 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.720664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.720839 (+ 175us) server_negotiation.cc:407] Connection header received
0928 18:00:20.720926 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.720926 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.720936 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.721010 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:20.721167 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.721617 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.722616 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.722986 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.723098 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.723108 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.723270 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.723307 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 18:00:20.723584 24391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.720545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:34921 (local address 127.5.46.66:45681)
0928 18:00:20.720741 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.720751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:20.720855 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:20.721012 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:20.721013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:20.721015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:20.721090 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.721092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.721718 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.721719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.722505 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:20.722507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.723102 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.723104 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:20.723195 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.723198 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:20.723265 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 18:00:20.723439 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":104,"threads_started":1}
I20260928 18:00:20.723599 24060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cda965fcc5a43338a36fadbf74bf853" candidate_uuid: "b614ef46fac741c09679ef42b45965ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c1389eb3de4067aa010c52f0048844" is_pre_election: true
I20260928 18:00:20.723783 24060 raft_consensus.cc:2454] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b614ef46fac741c09679ef42b45965ab in term 0.
I20260928 18:00:20.723976 24322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cda965fcc5a43338a36fadbf74bf853" candidate_uuid: "b614ef46fac741c09679ef42b45965ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d483625e4940c686c5a1d66e1630e6" is_pre_election: true
I20260928 18:00:20.724072 24124 leader_election.cc:302] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c1389eb3de4067aa010c52f0048844, b614ef46fac741c09679ef42b45965ab; no voters:
I20260928 18:00:20.724120 24322 raft_consensus.cc:2454] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b614ef46fac741c09679ef42b45965ab in term 0.
I20260928 18:00:20.724232 24386 raft_consensus.cc:2781] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:20.724318 24386 raft_consensus.cc:482] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:20.724352 24386 raft_consensus.cc:3037] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:20.725073 24386 raft_consensus.cc:504] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.725235 24386 leader_election.cc:288] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [CANDIDATE]: Term 1 election: Requested vote from peers 38d483625e4940c686c5a1d66e1630e6 (127.5.46.67:34921), 49c1389eb3de4067aa010c52f0048844 (127.5.46.65:41797)
I20260928 18:00:20.725391 24060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cda965fcc5a43338a36fadbf74bf853" candidate_uuid: "b614ef46fac741c09679ef42b45965ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c1389eb3de4067aa010c52f0048844"
I20260928 18:00:20.725405 24322 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cda965fcc5a43338a36fadbf74bf853" candidate_uuid: "b614ef46fac741c09679ef42b45965ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38d483625e4940c686c5a1d66e1630e6"
I20260928 18:00:20.725504 24322 raft_consensus.cc:3037] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:20.725509 24060 raft_consensus.cc:3037] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:20.726284 24322 raft_consensus.cc:2454] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b614ef46fac741c09679ef42b45965ab in term 1.
I20260928 18:00:20.726418 24060 raft_consensus.cc:2454] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b614ef46fac741c09679ef42b45965ab in term 1.
I20260928 18:00:20.726475 24125 leader_election.cc:302] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38d483625e4940c686c5a1d66e1630e6, b614ef46fac741c09679ef42b45965ab; no voters:
I20260928 18:00:20.726630 24386 raft_consensus.cc:2781] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:20.726790 24386 raft_consensus.cc:686] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 1 LEADER]: Becoming Leader. State: Replica: b614ef46fac741c09679ef42b45965ab, State: Running, Role: LEADER
I20260928 18:00:20.727008 24386 consensus_queue.cc:237] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } }
I20260928 18:00:20.728040 23918 catalog_manager.cc:5495] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab reported cstate change: term changed from 0 to 1, leader changed from <none> to b614ef46fac741c09679ef42b45965ab (127.5.46.66). New cstate: current_term: 1 leader_uuid: "b614ef46fac741c09679ef42b45965ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b614ef46fac741c09679ef42b45965ab" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 45529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 } health_report { overall_health: UNKNOWN } } }
I20260928 18:00:20.742882 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.740440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57692 (local address 127.5.46.66:45529)
0928 18:00:20.740510 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:20.740512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:20.740534 (+ 22us) server_negotiation.cc:407] Connection header received
0928 18:00:20.740570 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:20.740570 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:20.740590 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:20.740653 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:20.740861 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.741358 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.742183 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:20.742395 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:20.742454 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:20.742592 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:20.742678 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:20.742755 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:20.742796 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 18:00:20.742806 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 18:00:20.744977 24060 raft_consensus.cc:1261] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 1 FOLLOWER]: Refusing update from remote peer b614ef46fac741c09679ef42b45965ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:20.744977 24322 raft_consensus.cc:1261] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 1 FOLLOWER]: Refusing update from remote peer b614ef46fac741c09679ef42b45965ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:20.745388 24392 consensus_queue.cc:1035] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c1389eb3de4067aa010c52f0048844" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:20.745527 24386 consensus_queue.cc:1035] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "38d483625e4940c686c5a1d66e1630e6" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 34921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:20.748039 24395 mvcc.cc:205] Tried to move back new op lower bound from 7334373051368366080 to 7334373051298521088. Current Snapshot: MvccSnapshot[applied={T|T < 7334373051368366080}]
I20260928 18:00:20.748430 24396 mvcc.cc:205] Tried to move back new op lower bound from 7334373051368366080 to 7334373051298521088. Current Snapshot: MvccSnapshot[applied={T|T < 7334373051368366080}]
I20260928 18:00:20.748974 24397 mvcc.cc:205] Tried to move back new op lower bound from 7334373051368366080 to 7334373051298521088. Current Snapshot: MvccSnapshot[applied={T|T < 7334373051368366080}]
I20260928 18:00:20.750726 23918 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:20.750809 23918 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:20.751691 23918 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e616c50e5164ec09e8bd10c2dd1ab4f: Sending DeleteTablet for 3 replicas of tablet 2cda965fcc5a43338a36fadbf74bf853
I20260928 18:00:20.752080 24040 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:20 UTC) from {username='slave'} at 127.0.0.1:58890
I20260928 18:00:20.752080 24302 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:20 UTC) from {username='slave'} at 127.0.0.1:52688
I20260928 18:00:20.752341 24407 tablet_replica.cc:330] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: stopping tablet replica
I20260928 18:00:20.752455 24407 raft_consensus.cc:2229] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:20.752506 24171 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cda965fcc5a43338a36fadbf74bf853 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:20 UTC) from {username='slave'} at 127.0.0.1:57686
I20260928 18:00:20.752581 24407 raft_consensus.cc:2258] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:20.752753 24406 tablet_replica.cc:330] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: stopping tablet replica
I20260928 18:00:20.752784 24408 tablet_replica.cc:330] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: stopping tablet replica
I20260928 18:00:20.752854 24406 raft_consensus.cc:2229] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:20.752964 24406 raft_consensus.cc:2258] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:20.752983 24408 raft_consensus.cc:2229] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:20.753165 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23978
I20260928 18:00:20.753243 24407 ts_tablet_manager.cc:1905] T 2cda965fcc5a43338a36fadbf74bf853 P 49c1389eb3de4067aa010c52f0048844: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:20.753772 24408 raft_consensus.cc:2258] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:20.755079 24408 ts_tablet_manager.cc:1905] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:20.756129 24406 ts_tablet_manager.cc:1905] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:20.757306 24408 ts_tablet_manager.cc:1918] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:20.757395 24408 log.cc:1196] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/2cda965fcc5a43338a36fadbf74bf853
I20260928 18:00:20.757665 24408 ts_tablet_manager.cc:1939] T 2cda965fcc5a43338a36fadbf74bf853 P b614ef46fac741c09679ef42b45965ab: Deleting consensus metadata
I20260928 18:00:20.757915 24406 ts_tablet_manager.cc:1918] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:20.758013 24406 log.cc:1196] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/2cda965fcc5a43338a36fadbf74bf853
I20260928 18:00:20.758311 24406 ts_tablet_manager.cc:1939] T 2cda965fcc5a43338a36fadbf74bf853 P 38d483625e4940c686c5a1d66e1630e6: Deleting consensus metadata
I20260928 18:00:20.758397 23904 catalog_manager.cc:4841] TS b614ef46fac741c09679ef42b45965ab (127.5.46.66:45529): tablet 2cda965fcc5a43338a36fadbf74bf853 (table test-table [id=c3e99b62df5349bca0f9f200f2de3edd]) successfully deleted
W20260928 18:00:20.758965 23902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:41797 (error 108)
I20260928 18:00:20.759002 23903 catalog_manager.cc:4841] TS 38d483625e4940c686c5a1d66e1630e6 (127.5.46.67:34921): tablet 2cda965fcc5a43338a36fadbf74bf853 (table test-table [id=c3e99b62df5349bca0f9f200f2de3edd]) successfully deleted
I20260928 18:00:20.759160 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24109
I20260928 18:00:20.760123 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:20.759379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41797 (local address 127.0.0.1:58904)
0928 18:00:20.760017 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:20.760076 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":609}
W20260928 18:00:20.760310 23902 catalog_manager.cc:4584] TS 49c1389eb3de4067aa010c52f0048844 (127.5.46.65:41797): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2cda965fcc5a43338a36fadbf74bf853: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41797: connect: Connection refused (error 111)
I20260928 18:00:20.764518 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24240
I20260928 18:00:20.769586 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 23887
2026-09-28T18:00:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (776 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:20 dist-test-slave-z2t3 krb5kdc[24412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:20 dist-test-slave-z2t3 krb5kdc[24412](info): set up 2 sockets
Sep 28 18:00:20 dist-test-slave-z2t3 krb5kdc[24412](info): commencing operation
krb5kdc: starting...
W20260928 18:00:21.811939 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:21 dist-test-slave-z2t3 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:21.939998 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38351
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:22.024493 24425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:22.024713 24425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:22.024736 24425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:22.025969 24425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:22.026019 24425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:22.026034 24425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:22.026047 24425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:22.026069 24425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:22.027663 24425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:22.027971 24425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:22.028237 24425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 18:00:22.033339 24425 server_base.cc:1034] running on GCE node
W20260928 18:00:22.033303 24433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.033319 24431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.033294 24430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.033816 24425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:22.034134 24425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:22.035328 24425 hybrid_clock.cc:648] HybridClock initialized: now 1790618422035321 us; error 38 us; skew 500 ppm
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:22.038049 24425 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:22.038514 24425 webserver.cc:460] Webserver started at http://127.5.46.126:37079/ using document root <none> and password file <none>
I20260928 18:00:22.038803 24425 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:22.038882 24425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:22.039037 24425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:22.040222 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "68cbf281eae04dc0a16a5db2fb36f74c"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1afda628a1612053a3100a1c31faeea0"
server_key_iv: "e94dccf6ecf0d69dc0b9989f977bd10d"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.040628 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "68cbf281eae04dc0a16a5db2fb36f74c"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1afda628a1612053a3100a1c31faeea0"
server_key_iv: "e94dccf6ecf0d69dc0b9989f977bd10d"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.042099 24425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:22.042938 24440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.043156 24425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:22.043275 24425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "68cbf281eae04dc0a16a5db2fb36f74c"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1afda628a1612053a3100a1c31faeea0"
server_key_iv: "e94dccf6ecf0d69dc0b9989f977bd10d"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.043354 24425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:22.066807 24425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:22.069242 24425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:22.069424 24425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:22.073869 24425 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:38351
I20260928 18:00:22.073897 24492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:38351 every 8 connection(s)
I20260928 18:00:22.074306 24425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:22.075261 24493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:22.075363 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24425
I20260928 18:00:22.075489 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:22.075711 5305 external_mini_cluster.cc:1428] Setting key 30d78c028b4b0a79893a20361bd0c48a
I20260928 18:00:22.078991 24493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: Bootstrap starting.
I20260928 18:00:22.079730 24493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:22.080085 24493 log.cc:824] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:22.080868 24493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: No bootstrap required, opened a new log
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:22.083232 24493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } }
I20260928 18:00:22.083432 24493 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:22.083494 24493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68cbf281eae04dc0a16a5db2fb36f74c, State: Initialized, Role: FOLLOWER
I20260928 18:00:22.083636 24493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } }
I20260928 18:00:22.083699 24493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:22.083721 24493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:22.083789 24493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:22.084527 24493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } }
I20260928 18:00:22.084689 24493 leader_election.cc:302] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68cbf281eae04dc0a16a5db2fb36f74c; no voters:
I20260928 18:00:22.084905 24493 leader_election.cc:288] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:22.084975 24498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:22.085162 24498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [term 1 LEADER]: Becoming Leader. State: Replica: 68cbf281eae04dc0a16a5db2fb36f74c, State: Running, Role: LEADER
I20260928 18:00:22.085189 24493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:22.085253 24498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } }
I20260928 18:00:22.085595 24499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68cbf281eae04dc0a16a5db2fb36f74c. Latest consensus state: current_term: 1 leader_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } } }
I20260928 18:00:22.085770 24499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:22.086247 24507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:22.086606 24500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cbf281eae04dc0a16a5db2fb36f74c" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 38351 } } }
I20260928 18:00:22.086695 24500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:22.087191 24507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:22.088122 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.076412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34170 (local address 127.5.46.126:38351)
0928 18:00:22.076632 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.076642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.076688 (+ 46us) server_negotiation.cc:407] Connection header received
0928 18:00:22.077197 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.077208 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.077277 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.077441 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.078181 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.079565 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.080489 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.080991 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.081136 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.083984 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.083988 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.083989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:22.084000 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.086320 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.086879 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.086882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.086883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.086995 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.087240 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.087240 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.087241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.087511 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.087684 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.087762 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.087954 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":96,"threads_started":1}
I20260928 18:00:22.088819 24507 catalog_manager.cc:1347] Generated new cluster ID: fa28ff50a6b247de92b5d3d3045b3d03
I20260928 18:00:22.088865 24507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:22.126186 24507 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:22.126964 24507 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:22.136829 24507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: Generated new TSK 0
I20260928 18:00:22.137113 24507 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.46.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
Entry for principal kudu/127.5.46.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab.
I20260928 18:00:22.179282 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:22.264379 24519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:22.264595 24519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:22.264621 24519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:22.265791 24519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:22.265828 24519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:22.265854 24519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:22.267262 24519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.65.keytab
--principal=kudu/127.5.46.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:22.267473 24519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:22.267657 24519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:22.272917 24527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.272939 24525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.272917 24524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.273317 24519 server_base.cc:1034] running on GCE node
I20260928 18:00:22.273571 24519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:22.273797 24519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:22.274940 24519 hybrid_clock.cc:648] HybridClock initialized: now 1790618422274919 us; error 37 us; skew 500 ppm
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:22.277499 24519 init.cc:377] Logged in from keytab as kudu/127.5.46.65@KRBTEST.COM (short username kudu)
I20260928 18:00:22.277959 24519 webserver.cc:460] Webserver started at http://127.5.46.65:37911/ using document root <none> and password file <none>
I20260928 18:00:22.278214 24519 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:22.278282 24519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:22.278426 24519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:22.279482 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "49024825c3224bba8ee68e69d679f8f0"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "8f7aa0fd77030c462dcd04412fdbde26"
server_key_iv: "65553fbb011928893e43d67571338125"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.279891 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "49024825c3224bba8ee68e69d679f8f0"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "8f7aa0fd77030c462dcd04412fdbde26"
server_key_iv: "65553fbb011928893e43d67571338125"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.281351 24519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:22.282109 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.282276 24519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:22.282395 24519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "49024825c3224bba8ee68e69d679f8f0"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "8f7aa0fd77030c462dcd04412fdbde26"
server_key_iv: "65553fbb011928893e43d67571338125"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.282471 24519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:22.309803 24519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:22.312371 24519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:22.312567 24519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:22.312835 24519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:22.313172 24519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:22.313226 24519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.313261 24519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:22.313289 24519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.319465 24519 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:44211
I20260928 18:00:22.319510 24647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:44211 every 8 connection(s)
I20260928 18:00:22.319881 24519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 18:00:22.324904 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24519
I20260928 18:00:22.325058 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:22.325256 5305 external_mini_cluster.cc:1428] Setting key a5508ad75d29266c07e72e6b05f1f40c
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.65@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:22.331236 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.320767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:37879 (local address 127.5.46.126:38351)
0928 18:00:22.320818 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.320820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.320989 (+ 169us) server_negotiation.cc:407] Connection header received
0928 18:00:22.321560 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.321562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.321574 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.321634 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.322658 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.323179 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.324297 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.324524 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.324616 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.327529 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.327531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.327531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:22.327539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.329432 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.329927 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.329928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.329929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.329996 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.330401 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.330402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.330403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.330565 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.330648 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.330994 (+ 346us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.331098 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 18:00:22.331266 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.320682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38351 (local address 127.5.46.65:37879)
0928 18:00:22.320837 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.320885 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.321439 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.321722 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.321727 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.322325 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:22.322540 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.322546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.323282 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.323283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.324198 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.324200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.324629 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.324630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.324775 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.325399 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:22.325409 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:22.327412 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:22.329529 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.329537 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.329539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.329841 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.330099 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.330100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.330100 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.330317 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.330662 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:22.330668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:22.330836 (+ 168us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.330939 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.331080 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260928 18:00:22.331903 24648 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38351
I20260928 18:00:22.332024 24648 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:22.332249 24648 heartbeater.cc:511] Master 127.5.46.126:38351 requested a full tablet report, sending...
I20260928 18:00:22.332796 24457 ts_manager.cc:194] Registered new tserver with Master: 49024825c3224bba8ee68e69d679f8f0 (127.5.46.65:44211)
I20260928 18:00:22.333422 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.65@KRBTEST.COM'} at 127.5.46.65:37879
WARNING: no policy specified for kudu/127.5.46.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
Entry for principal kudu/127.5.46.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab.
I20260928 18:00:22.356383 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:22.438773 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:22.439000 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:22.439033 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:22.440337 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:22.440390 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:22.440420 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:22.441987 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.66.keytab
--principal=kudu/127.5.46.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:22.442260 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:22.442540 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:22.447576 24659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.447633 24658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.447576 24661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.447961 24653 server_base.cc:1034] running on GCE node
I20260928 18:00:22.448221 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:22.448485 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:22.449754 24653 hybrid_clock.cc:648] HybridClock initialized: now 1790618422449753 us; error 45 us; skew 500 ppm
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:22.452658 24653 init.cc:377] Logged in from keytab as kudu/127.5.46.66@KRBTEST.COM (short username kudu)
I20260928 18:00:22.453121 24653 webserver.cc:460] Webserver started at http://127.5.46.66:35903/ using document root <none> and password file <none>
I20260928 18:00:22.453387 24653 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:22.453462 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:22.453612 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:22.454758 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "876ea93ff3b0484ca5240f4b60c4e18d"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "c898dc3105015b3503a86e7aa91226c8"
server_key_iv: "14984fa63d68277ddfdafeea8f2fe963"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.455112 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "876ea93ff3b0484ca5240f4b60c4e18d"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "c898dc3105015b3503a86e7aa91226c8"
server_key_iv: "14984fa63d68277ddfdafeea8f2fe963"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.462541 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.003s
I20260928 18:00:22.463758 24668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.463985 24653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:22.464092 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "876ea93ff3b0484ca5240f4b60c4e18d"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "c898dc3105015b3503a86e7aa91226c8"
server_key_iv: "14984fa63d68277ddfdafeea8f2fe963"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.464186 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:22.481854 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:22.484059 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:22.484184 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:22.484380 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:22.484704 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:22.484745 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.484771 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:22.484818 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.491539 24653 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:40423
I20260928 18:00:22.491578 24781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:40423 every 8 connection(s)
I20260928 18:00:22.492017 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:22.492705 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24653
I20260928 18:00:22.492861 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:22.493050 5305 external_mini_cluster.cc:1428] Setting key e2b2f61b2f2b711f2982445083380ce2
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.66@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.46.67@KRBTEST.COM; defaulting to no policy
I20260928 18:00:22.502727 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.492807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:33509 (local address 127.5.46.126:38351)
0928 18:00:22.492870 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.492872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.493148 (+ 276us) server_negotiation.cc:407] Connection header received
0928 18:00:22.493796 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.493797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.493810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.493866 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.494841 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.495478 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.496633 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.496850 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.496931 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.499440 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.499442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.499442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:22.499451 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.501144 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.501579 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.501580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.501581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.501650 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.501963 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.501965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.501965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.502095 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.502174 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.502505 (+ 331us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.502605 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 18:00:22.503036 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.492773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38351 (local address 127.5.46.66:33509)
0928 18:00:22.492968 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.493013 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.493616 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.493940 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.493944 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.494524 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:22.494733 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.494737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.495599 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.495600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.496523 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.496525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.496939 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.496940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.497079 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.497661 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:22.497671 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:22.499333 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:22.501238 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.501247 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.501250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.501497 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.501738 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.501739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.501739 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.501902 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.502184 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:22.502192 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:22.502352 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.502485 (+ 133us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.502718 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.67@KRBTEST.COM" created.
I20260928 18:00:22.503818 24782 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38351
I20260928 18:00:22.503993 24782 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:22.504236 24782 heartbeater.cc:511] Master 127.5.46.126:38351 requested a full tablet report, sending...
I20260928 18:00:22.504719 24457 ts_manager.cc:194] Registered new tserver with Master: 876ea93ff3b0484ca5240f4b60c4e18d (127.5.46.66:40423)
I20260928 18:00:22.505174 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.66@KRBTEST.COM'} at 127.5.46.66:33509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
Entry for principal kudu/127.5.46.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab.
I20260928 18:00:22.519729 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:22.605067 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:22.605285 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:22.605314 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:22.606477 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:22.606518 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:22.606544 24787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:22.608059 24787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.67.keytab
--principal=kudu/127.5.46.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:22.608269 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:22.608456 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:22.613791 24792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.613885 24787 server_base.cc:1034] running on GCE node
W20260928 18:00:22.613802 24793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.613802 24795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.614292 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:22.614555 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:22.615733 24787 hybrid_clock.cc:648] HybridClock initialized: now 1790618422615714 us; error 45 us; skew 500 ppm
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:22.618280 24787 init.cc:377] Logged in from keytab as kudu/127.5.46.67@KRBTEST.COM (short username kudu)
I20260928 18:00:22.618764 24787 webserver.cc:460] Webserver started at http://127.5.46.67:42645/ using document root <none> and password file <none>
I20260928 18:00:22.619015 24787 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:22.619086 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:22.619246 24787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:22.620390 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "34ca2a23bfed46c4a8f0c327123596e9"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "e36b11f72e057c78d209646532e9f547"
server_key_iv: "8b1801d77e429bc02354a340d30c82cc"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.620795 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "34ca2a23bfed46c4a8f0c327123596e9"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "e36b11f72e057c78d209646532e9f547"
server_key_iv: "8b1801d77e429bc02354a340d30c82cc"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.622188 24787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:22.623178 24802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.623377 24787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:22.623487 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "34ca2a23bfed46c4a8f0c327123596e9"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "e36b11f72e057c78d209646532e9f547"
server_key_iv: "8b1801d77e429bc02354a340d30c82cc"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.623565 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:22.630491 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:22.632899 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:22.633082 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:22.633383 24787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:22.633814 24787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:22.633858 24787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.633896 24787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:22.633921 24787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.640645 24787 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:38053
I20260928 18:00:22.640698 24915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:38053 every 8 connection(s)
I20260928 18:00:22.641047 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:22.645360 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24787
I20260928 18:00:22.645442 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:22.645617 5305 external_mini_cluster.cc:1428] Setting key c9413bdd042f5652f8234e4f18c3df6d
Sep 28 18:00:22 dist-test-slave-z2t3 krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618422, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.67@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:22.651712 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.641897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58995 (local address 127.5.46.126:38351)
0928 18:00:22.641973 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.641976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.642154 (+ 178us) server_negotiation.cc:407] Connection header received
0928 18:00:22.642773 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.642774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.642787 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.642867 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.643961 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.644523 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.645779 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.646118 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.646197 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.648674 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.648676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.648676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:22.648682 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.650160 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.650587 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.650588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.650588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.650643 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.650926 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.650927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.650927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.651051 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.651124 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.651491 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.651573 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 18:00:22.651822 24918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.641768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:38351 (local address 127.5.46.67:58995)
0928 18:00:22.642008 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.642065 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.642633 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.642969 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.642977 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.643578 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:22.643789 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.643794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.644631 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.644633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.645636 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.645639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.646228 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.646229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.646381 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.646944 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:22.646952 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:22.648558 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:22.650258 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.650265 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.650267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.650514 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.650704 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:22.650705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:22.650705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:22.650809 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:22.651140 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:22.651146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:22.651325 (+ 179us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.651434 (+ 109us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.651577 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":114,"threads_started":1}
I20260928 18:00:22.652444 24916 heartbeater.cc:347] Connected to a master server at 127.5.46.126:38351
I20260928 18:00:22.652567 24916 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:22.652810 24916 heartbeater.cc:511] Master 127.5.46.126:38351 requested a full tablet report, sending...
I20260928 18:00:22.653265 24457 ts_manager.cc:194] Registered new tserver with Master: 34ca2a23bfed46c4a8f0c327123596e9 (127.5.46.67:38053)
I20260928 18:00:22.653666 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.46.67@KRBTEST.COM'} at 127.5.46.67:58995
I20260928 18:00:22.657459 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:22.665527 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.658999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.5.46.126:38351)
0928 18:00:22.659086 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.659088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.659206 (+ 118us) server_negotiation.cc:407] Connection header received
0928 18:00:22.659295 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.659296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.659315 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.659381 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.660041 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.660531 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.661277 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.661467 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.661555 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.662905 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.662907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.662907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:22.662917 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.664270 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.664644 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.664645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.664646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.664707 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:22.664920 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:22.664920 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:22.664921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:22.665088 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.665196 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.665329 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.665420 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 18:00:22.667237 24457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:22.669140 24457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:22.676220 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:40423 (local address 127.0.0.1:59930)
0928 18:00:22.672373 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.672419 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.672499 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.672817 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.672820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.672822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:22.672896 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.672899 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.673837 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.673838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.675052 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.675054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.675879 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.675880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.675961 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.675979 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.676062 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.676120 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
I20260928 18:00:22.676226 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59930 (local address 127.5.46.66:40423)
0928 18:00:22.672428 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.672434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.672482 (+ 48us) server_negotiation.cc:407] Connection header received
0928 18:00:22.672610 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.672613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.672632 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.672706 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:22.672998 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.673725 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.675150 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.675756 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.675832 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.675927 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.676091 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.676133 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":84,"threads_started":1}
I20260928 18:00:22.676971 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:38053 (local address 127.0.0.1:55396)
0928 18:00:22.672531 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.672540 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.673007 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.673211 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.673212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.673213 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:22.673264 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.673265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.674359 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.674360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.675534 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.675537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.676711 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.676712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.676776 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.676781 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.676854 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.676895 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260928 18:00:22.676998 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.5.46.67:38053)
0928 18:00:22.673049 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.673053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.673063 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:22.673099 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.673101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.673115 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.673214 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:22.673330 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.674227 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.675640 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.676424 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.676502 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.676552 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.676872 (+ 320us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.676917 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":88,"threads_started":1}
I20260928 18:00:22.677223 24716 tablet_service.cc:1467] Processing CreateTablet for tablet c1ca67541d794c49b0d9e186d48828a3 (DEFAULT_TABLE table=test-table [id=f9d4071300d54593bfdbcdb009cee1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:22.677377 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:44211 (local address 127.0.0.1:43574)
0928 18:00:22.672418 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.672430 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.672499 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.672866 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.672866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.672867 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:22.672901 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.672902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.674884 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.674885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.675945 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.675947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.677148 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.677149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.677212 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.677215 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.677266 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.677317 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":163,"threads_started":1}
I20260928 18:00:22.677639 24716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ca67541d794c49b0d9e186d48828a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:22.677645 24850 tablet_service.cc:1467] Processing CreateTablet for tablet c1ca67541d794c49b0d9e186d48828a3 (DEFAULT_TABLE table=test-table [id=f9d4071300d54593bfdbcdb009cee1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:22.677894 24850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ca67541d794c49b0d9e186d48828a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:22.680503 24934 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Bootstrap starting.
I20260928 18:00:22.680667 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.672246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43574 (local address 127.5.46.65:44211)
0928 18:00:22.672650 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.672656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.672665 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:22.672703 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.672705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.672720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.672790 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:22.673855 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.674763 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.676044 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.677036 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.677139 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.677195 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.680511 (+ 3316us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.680580 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":82,"threads_started":1}
I20260928 18:00:22.680881 24933 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Bootstrap starting.
I20260928 18:00:22.681283 24934 tablet_bootstrap.cc:654] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:22.681419 24933 tablet_bootstrap.cc:654] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:22.681424 24582 tablet_service.cc:1467] Processing CreateTablet for tablet c1ca67541d794c49b0d9e186d48828a3 (DEFAULT_TABLE table=test-table [id=f9d4071300d54593bfdbcdb009cee1de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:22.681798 24582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ca67541d794c49b0d9e186d48828a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:22.681850 24933 log.cc:824] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:22.681890 24934 log.cc:824] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:22.682617 24933 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: No bootstrap required, opened a new log
I20260928 18:00:22.682652 24934 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: No bootstrap required, opened a new log
I20260928 18:00:22.682667 24933 ts_tablet_manager.cc:1397] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:22.682714 24934 ts_tablet_manager.cc:1397] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:22.684285 24933 raft_consensus.cc:348] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.684419 24933 raft_consensus.cc:374] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:22.684440 24933 raft_consensus.cc:729] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34ca2a23bfed46c4a8f0c327123596e9, State: Initialized, Role: FOLLOWER
I20260928 18:00:22.684594 24933 consensus_queue.cc:260] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.684913 24933 ts_tablet_manager.cc:1428] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:22.684877 24934 raft_consensus.cc:348] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.685071 24916 heartbeater.cc:503] Master 127.5.46.126:38351 was elected leader, sending a full tablet report...
I20260928 18:00:22.685067 24934 raft_consensus.cc:374] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:22.685118 24934 raft_consensus.cc:729] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876ea93ff3b0484ca5240f4b60c4e18d, State: Initialized, Role: FOLLOWER
I20260928 18:00:22.685240 24934 consensus_queue.cc:260] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.685519 24934 ts_tablet_manager.cc:1428] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:22.685665 24782 heartbeater.cc:503] Master 127.5.46.126:38351 was elected leader, sending a full tablet report...
I20260928 18:00:22.685922 24939 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: Bootstrap starting.
I20260928 18:00:22.686679 24939 tablet_bootstrap.cc:654] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:22.687202 24939 log.cc:824] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:22.688109 24939 tablet_bootstrap.cc:492] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: No bootstrap required, opened a new log
I20260928 18:00:22.688181 24939 ts_tablet_manager.cc:1397] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:22.689674 24939 raft_consensus.cc:348] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.689857 24939 raft_consensus.cc:374] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:22.689883 24939 raft_consensus.cc:729] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49024825c3224bba8ee68e69d679f8f0, State: Initialized, Role: FOLLOWER
I20260928 18:00:22.690045 24939 consensus_queue.cc:260] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.690384 24939 ts_tablet_manager.cc:1428] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 18:00:22.690429 24648 heartbeater.cc:503] Master 127.5.46.126:38351 was elected leader, sending a full tablet report...
I20260928 18:00:22.699723 24938 raft_consensus.cc:482] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:22.699970 24938 raft_consensus.cc:504] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.700345 24938 leader_election.cc:288] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49024825c3224bba8ee68e69d679f8f0 (127.5.46.65:44211), 34ca2a23bfed46c4a8f0c327123596e9 (127.5.46.67:38053)
I20260928 18:00:22.704449 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.700616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48867 (local address 127.5.46.65:44211)
0928 18:00:22.700729 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.700732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.700760 (+ 28us) server_negotiation.cc:407] Connection header received
0928 18:00:22.700815 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.700816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.700828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.700882 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:22.701045 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.701826 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.703428 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.704050 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.704137 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.704153 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.704333 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.704377 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 18:00:22.704449 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.700613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:37377 (local address 127.5.46.67:38053)
0928 18:00:22.700713 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.700715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.700727 (+ 12us) server_negotiation.cc:407] Connection header received
0928 18:00:22.700759 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.700760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.700774 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.700855 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:22.701046 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.701803 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.703429 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.704102 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.704169 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.704178 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.704333 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.704374 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260928 18:00:22.704480 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.700512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:38053 (local address 127.5.46.66:37377)
0928 18:00:22.700575 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.700586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.700631 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.700875 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.700876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.700878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:22.700957 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.700960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.701929 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.701930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.703325 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.703328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.704171 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.704171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.704239 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.704241 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.704318 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.704355 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":110}
I20260928 18:00:22.704488 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.700528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:44211 (local address 127.5.46.66:48867)
0928 18:00:22.700701 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.700712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:22.700764 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:22.700896 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:22.700897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:22.700898 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:22.700972 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.700973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.701929 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.701930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.703323 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:22.703328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.704150 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.704152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:22.704230 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.704235 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:22.704299 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 18:00:22.704350 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":63,"thread_start_us":117,"threads_started":1}
I20260928 18:00:22.704921 24870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ca67541d794c49b0d9e186d48828a3" candidate_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ca2a23bfed46c4a8f0c327123596e9" is_pre_election: true
I20260928 18:00:22.704988 24602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ca67541d794c49b0d9e186d48828a3" candidate_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49024825c3224bba8ee68e69d679f8f0" is_pre_election: true
I20260928 18:00:22.705101 24870 raft_consensus.cc:2454] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 876ea93ff3b0484ca5240f4b60c4e18d in term 0.
I20260928 18:00:22.705142 24602 raft_consensus.cc:2454] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 876ea93ff3b0484ca5240f4b60c4e18d in term 0.
I20260928 18:00:22.705358 24669 leader_election.cc:302] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34ca2a23bfed46c4a8f0c327123596e9, 876ea93ff3b0484ca5240f4b60c4e18d; no voters:
I20260928 18:00:22.705507 24938 raft_consensus.cc:2781] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:22.705561 24938 raft_consensus.cc:482] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:22.705597 24938 raft_consensus.cc:3037] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:22.706327 24938 raft_consensus.cc:504] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.706476 24938 leader_election.cc:288] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [CANDIDATE]: Term 1 election: Requested vote from peers 49024825c3224bba8ee68e69d679f8f0 (127.5.46.65:44211), 34ca2a23bfed46c4a8f0c327123596e9 (127.5.46.67:38053)
I20260928 18:00:22.706655 24602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ca67541d794c49b0d9e186d48828a3" candidate_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49024825c3224bba8ee68e69d679f8f0"
I20260928 18:00:22.706773 24602 raft_consensus.cc:3037] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:22.707485 24602 raft_consensus.cc:2454] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 876ea93ff3b0484ca5240f4b60c4e18d in term 1.
I20260928 18:00:22.707630 24870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ca67541d794c49b0d9e186d48828a3" candidate_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34ca2a23bfed46c4a8f0c327123596e9"
I20260928 18:00:22.707676 24672 leader_election.cc:302] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49024825c3224bba8ee68e69d679f8f0, 876ea93ff3b0484ca5240f4b60c4e18d; no voters:
I20260928 18:00:22.707719 24870 raft_consensus.cc:3037] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:22.707844 24938 raft_consensus.cc:2781] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:22.708061 24938 raft_consensus.cc:686] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 1 LEADER]: Becoming Leader. State: Replica: 876ea93ff3b0484ca5240f4b60c4e18d, State: Running, Role: LEADER
I20260928 18:00:22.708169 24938 consensus_queue.cc:237] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } }
I20260928 18:00:22.708686 24870 raft_consensus.cc:2454] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 876ea93ff3b0484ca5240f4b60c4e18d in term 1.
I20260928 18:00:22.709081 24455 catalog_manager.cc:5495] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d reported cstate change: term changed from 0 to 1, leader changed from <none> to 876ea93ff3b0484ca5240f4b60c4e18d (127.5.46.66). New cstate: current_term: 1 leader_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "876ea93ff3b0484ca5240f4b60c4e18d" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 40423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 } health_report { overall_health: UNKNOWN } } }
I20260928 18:00:22.715622 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.713145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59936 (local address 127.5.46.66:40423)
0928 18:00:22.713216 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.713217 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.713225 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:22.713263 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.713264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.713284 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.713365 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:22.713515 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.713989 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.714763 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.714998 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.715079 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.715254 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:22.715387 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:22.715489 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.715508 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.715530 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 18:00:22.717967 24602 raft_consensus.cc:1261] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 1 FOLLOWER]: Refusing update from remote peer 876ea93ff3b0484ca5240f4b60c4e18d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:22.718127 24870 raft_consensus.cc:1261] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 1 FOLLOWER]: Refusing update from remote peer 876ea93ff3b0484ca5240f4b60c4e18d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:22.718357 24943 consensus_queue.cc:1035] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [LEADER]: Connected to new peer: Peer: permanent_uuid: "49024825c3224bba8ee68e69d679f8f0" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:22.718468 24938 consensus_queue.cc:1035] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [LEADER]: Connected to new peer: Peer: permanent_uuid: "34ca2a23bfed46c4a8f0c327123596e9" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:22.722097 24947 mvcc.cc:205] Tried to move back new op lower bound from 7334373059449663488 to 7334373059413413888. Current Snapshot: MvccSnapshot[applied={T|T < 7334373059449663488}]
I20260928 18:00:22.724264 24455 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:22.724368 24455 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:22.725278 24455 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68cbf281eae04dc0a16a5db2fb36f74c: Sending DeleteTablet for 3 replicas of tablet c1ca67541d794c49b0d9e186d48828a3
I20260928 18:00:22.725647 24582 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ca67541d794c49b0d9e186d48828a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:43574
I20260928 18:00:22.725767 24850 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ca67541d794c49b0d9e186d48828a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:55396
I20260928 18:00:22.725935 24955 tablet_replica.cc:330] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0: stopping tablet replica
I20260928 18:00:22.725901 24716 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ca67541d794c49b0d9e186d48828a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:22 UTC) from {username='kudu', principal='kudu/127.5.46.126@KRBTEST.COM'} at 127.0.0.1:59930
I20260928 18:00:22.725999 24956 tablet_replica.cc:330] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: stopping tablet replica
I20260928 18:00:22.726087 24955 raft_consensus.cc:2229] T c1ca67541d794c49b0d9e186d48828a3 P 49024825c3224bba8ee68e69d679f8f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:22.726147 24956 raft_consensus.cc:2229] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:22.726295 24956 raft_consensus.cc:2258] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:22.726366 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24519
I20260928 18:00:22.726418 24957 tablet_replica.cc:330] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: stopping tablet replica
I20260928 18:00:22.727188 24956 ts_tablet_manager.cc:1905] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:22.727797 24957 raft_consensus.cc:2229] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:22.727983 24957 raft_consensus.cc:2258] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:22.728480 24957 ts_tablet_manager.cc:1905] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:22.729405 24956 ts_tablet_manager.cc:1918] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:22.729477 24956 log.cc:1196] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/c1ca67541d794c49b0d9e186d48828a3
I20260928 18:00:22.729785 24956 ts_tablet_manager.cc:1939] T c1ca67541d794c49b0d9e186d48828a3 P 34ca2a23bfed46c4a8f0c327123596e9: Deleting consensus metadata
I20260928 18:00:22.730242 24441 catalog_manager.cc:4841] TS 34ca2a23bfed46c4a8f0c327123596e9 (127.5.46.67:38053): tablet c1ca67541d794c49b0d9e186d48828a3 (table test-table [id=f9d4071300d54593bfdbcdb009cee1de]) successfully deleted
I20260928 18:00:22.731364 24957 ts_tablet_manager.cc:1918] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:22.731462 24957 log.cc:1196] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/c1ca67541d794c49b0d9e186d48828a3
I20260928 18:00:22.731812 24957 ts_tablet_manager.cc:1939] T c1ca67541d794c49b0d9e186d48828a3 P 876ea93ff3b0484ca5240f4b60c4e18d: Deleting consensus metadata
I20260928 18:00:22.732369 24441 catalog_manager.cc:4841] TS 876ea93ff3b0484ca5240f4b60c4e18d (127.5.46.66:40423): tablet c1ca67541d794c49b0d9e186d48828a3 (table test-table [id=f9d4071300d54593bfdbcdb009cee1de]) successfully deleted
W20260928 18:00:22.734858 24444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:44211 (error 108)
I20260928 18:00:22.735505 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.735338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:44211 (local address 127.0.0.1:43576)
0928 18:00:22.735414 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:22.735460 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:44211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 18:00:22.735565 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24653
W20260928 18:00:22.736336 24444 catalog_manager.cc:4584] TS 49024825c3224bba8ee68e69d679f8f0 (127.5.46.65:44211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1ca67541d794c49b0d9e186d48828a3: Network error: Client connection negotiation failed: client connection to 127.5.46.65:44211: connect: Connection refused (error 111)
I20260928 18:00:22.740042 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24787
I20260928 18:00:22.745430 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24425
2026-09-28T18:00:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1977 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T18:00:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:22Z Disabled control of system clock
I20260928 18:00:22.774042 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35191
--webserver_interface=127.5.46.126
--webserver_port=0
--builtin_ntp_servers=127.5.46.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:35191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:22.858762 24962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:22.858994 24962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:22.859017 24962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:22.860275 24962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:22.860324 24962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:22.860337 24962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:22.860350 24962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:22.860363 24962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:22.861794 24962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:35191
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:35191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:22.862077 24962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:22.862413 24962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:22.867528 24970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.867528 24968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:22.867714 24967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:22.867882 24962 server_base.cc:1034] running on GCE node
I20260928 18:00:22.868122 24962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:22.868399 24962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:22.869591 24962 hybrid_clock.cc:648] HybridClock initialized: now 1790618422869539 us; error 81 us; skew 500 ppm
I20260928 18:00:22.871148 24962 webserver.cc:460] Webserver started at http://127.5.46.126:44623/ using document root <none> and password file <none>
I20260928 18:00:22.871431 24962 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:22.871505 24962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:22.871645 24962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:22.872907 24962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "305aa59edde54e1a8939365b7c75d770"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1169dda5e5aee01e2393d62510151125"
server_key_iv: "cadec63c9493b4413872b1ea0154f88e"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.873303 24962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "305aa59edde54e1a8939365b7c75d770"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1169dda5e5aee01e2393d62510151125"
server_key_iv: "cadec63c9493b4413872b1ea0154f88e"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.874737 24962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:22.875613 24976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:22.875989 24962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:22.876096 24962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "305aa59edde54e1a8939365b7c75d770"
format_stamp: "Formatted at 2026-09-28 18:00:22 on dist-test-slave-z2t3"
server_key: "1169dda5e5aee01e2393d62510151125"
server_key_iv: "cadec63c9493b4413872b1ea0154f88e"
server_key_version: "encryptionkey@0"
I20260928 18:00:22.876192 24962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:22.897095 24962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:22.897475 24962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:22.897637 24962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:22.901938 24962 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:35191
I20260928 18:00:22.901973 25028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:35191 every 8 connection(s)
I20260928 18:00:22.902338 24962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:22.903491 25029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:22.906097 25029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: Bootstrap starting.
I20260928 18:00:22.906752 25029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:22.907006 25029 log.cc:824] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:22.907752 25029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: No bootstrap required, opened a new log
I20260928 18:00:22.909073 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 24962
I20260928 18:00:22.909219 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:22.909427 5305 external_mini_cluster.cc:1428] Setting key 3b43f78fcf84ca3409b9fc0f3a3f3b0f
I20260928 18:00:22.909605 25029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } }
I20260928 18:00:22.909785 25029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:22.909823 25029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305aa59edde54e1a8939365b7c75d770, State: Initialized, Role: FOLLOWER
I20260928 18:00:22.909948 25029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } }
I20260928 18:00:22.910043 25029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:22.910087 25029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:22.910132 25029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:22.910737 25029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } }
I20260928 18:00:22.910861 25029 leader_election.cc:302] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 305aa59edde54e1a8939365b7c75d770; no voters:
I20260928 18:00:22.911062 25029 leader_election.cc:288] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:22.911234 25034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:22.911377 25034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [term 1 LEADER]: Becoming Leader. State: Replica: 305aa59edde54e1a8939365b7c75d770, State: Running, Role: LEADER
I20260928 18:00:22.911427 25029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:22.911466 25034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } }
I20260928 18:00:22.911801 25034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 305aa59edde54e1a8939365b7c75d770. Latest consensus state: current_term: 1 leader_uuid: "305aa59edde54e1a8939365b7c75d770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } } }
I20260928 18:00:22.911948 25034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:22.911813 25035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "305aa59edde54e1a8939365b7c75d770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305aa59edde54e1a8939365b7c75d770" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 35191 } } }
I20260928 18:00:22.912073 25035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:22.912161 25041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:22.912731 25041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:22.914541 25041 catalog_manager.cc:1347] Generated new cluster ID: 803a2484d3d3407e96dd7af1fbf8f6f5
I20260928 18:00:22.914600 25041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:22.916110 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:22.909946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.5.46.126:35191)
0928 18:00:22.910274 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:22.910282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:22.910307 (+ 25us) server_negotiation.cc:407] Connection header received
0928 18:00:22.910411 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:22.910417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:22.910480 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:22.910628 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:22.911008 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.912415 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.913617 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:22.914223 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:22.914350 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:22.915238 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:22.915242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:22.915242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:22.915453 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:22.915498 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:22.915514 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:22.915585 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:22.915662 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 18:00:22.915894 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":99,"threads_started":1}
I20260928 18:00:22.923899 25041 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:22.924484 25041 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:22.932147 25041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: Generated new TSK 0
I20260928 18:00:22.932407 25041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:22.941258 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.65:0
--local_ip_for_outbound_sockets=127.5.46.65
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--builtin_ntp_servers=127.5.46.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:23.031189 25053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:23.031462 25053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:23.031507 25053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:23.032929 25053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:23.033020 25053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:23.033068 25053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.65
I20260928 18:00:23.035192 25053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.46.65
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.65
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:23.035513 25053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:23.035892 25053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:23.041100 25061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:23.041278 25053 server_base.cc:1034] running on GCE node
W20260928 18:00:23.041121 25058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:23.041136 25059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:23.041639 25053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:23.041918 25053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:23.043118 25053 hybrid_clock.cc:648] HybridClock initialized: now 1790618423043091 us; error 45 us; skew 500 ppm
I20260928 18:00:23.044591 25053 webserver.cc:460] Webserver started at http://127.5.46.65:38013/ using document root <none> and password file <none>
I20260928 18:00:23.044922 25053 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:23.045003 25053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:23.045173 25053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:23.046392 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data/instance:
uuid: "70afccbb4cff4855a5d9f1579ccbfcd1"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "0f4522282e98eb8b669028a7c18f9010"
server_key_iv: "e23cc849fb5d6f0753a7aa9bdcee1e9b"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.046788 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance:
uuid: "70afccbb4cff4855a5d9f1579ccbfcd1"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "0f4522282e98eb8b669028a7c18f9010"
server_key_iv: "e23cc849fb5d6f0753a7aa9bdcee1e9b"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.048369 25053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:23.049275 25067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.049525 25053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:23.049708 25053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal
uuid: "70afccbb4cff4855a5d9f1579ccbfcd1"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "0f4522282e98eb8b669028a7c18f9010"
server_key_iv: "e23cc849fb5d6f0753a7aa9bdcee1e9b"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.049811 25053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:23.066753 25053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:23.067153 25053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:23.067314 25053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:23.067571 25053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:23.068009 25053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:23.068070 25053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.068112 25053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:23.068138 25053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.074131 25053 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.65:41617
I20260928 18:00:23.074213 25180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.65:41617 every 8 connection(s)
I20260928 18:00:23.074586 25053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/data/info.pb
I20260928 18:00:23.077308 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25053
I20260928 18:00:23.077423 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-0/wal/instance
I20260928 18:00:23.077622 5305 external_mini_cluster.cc:1428] Setting key 256f080204b2c1a14cba028deba5ba3a
I20260928 18:00:23.078610 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.66:0
--local_ip_for_outbound_sockets=127.5.46.66
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--builtin_ntp_servers=127.5.46.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 18:00:23.080819 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.075431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.65:53151 (local address 127.5.46.126:35191)
0928 18:00:23.075526 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.075528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.075696 (+ 168us) server_negotiation.cc:407] Connection header received
0928 18:00:23.076268 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.076270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.076282 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.076344 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:23.077148 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.077929 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.079131 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.079400 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.079477 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.080524 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:23.080526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:23.080526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:23.080534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:23.080550 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:23.080555 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:23.080612 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.080703 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.080737 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 18:00:23.081051 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.075369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35191 (local address 127.5.46.65:53151)
0928 18:00:23.075562 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.075598 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.076146 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.076400 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.076403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.076411 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:23.076943 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.076948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.078041 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.078042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.079040 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.079043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.079486 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.079487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.079656 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.080353 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:23.080353 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:23.080396 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:23.080430 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:23.080612 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:23.080619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:23.080622 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.080687 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.080803 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260928 18:00:23.081602 25181 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35191
I20260928 18:00:23.081722 25181 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:23.081935 25181 heartbeater.cc:511] Master 127.5.46.126:35191 requested a full tablet report, sending...
I20260928 18:00:23.082474 24993 ts_manager.cc:194] Registered new tserver with Master: 70afccbb4cff4855a5d9f1579ccbfcd1 (127.5.46.65:41617)
I20260928 18:00:23.083108 24993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.65:53151
W20260928 18:00:23.163295 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:23.163519 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:23.163542 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:23.164860 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:23.164911 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:23.164937 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.66
I20260928 18:00:23.166337 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.46.66
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.66
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:23.166581 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:23.166886 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:23.172076 25190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:23.172070 25189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:23.172070 25192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:23.172287 25184 server_base.cc:1034] running on GCE node
I20260928 18:00:23.172521 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:23.172812 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:23.173966 25184 hybrid_clock.cc:648] HybridClock initialized: now 1790618423173952 us; error 29 us; skew 500 ppm
I20260928 18:00:23.175297 25184 webserver.cc:460] Webserver started at http://127.5.46.66:32821/ using document root <none> and password file <none>
I20260928 18:00:23.175539 25184 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:23.175601 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:23.175833 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:23.176963 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data/instance:
uuid: "b321187b0fd4475faf42154b4c2eda88"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "e60566880f156840b26c63fc55e2073e"
server_key_iv: "a8eaaef316df9b04c21a93dc10fcc27e"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.177325 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance:
uuid: "b321187b0fd4475faf42154b4c2eda88"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "e60566880f156840b26c63fc55e2073e"
server_key_iv: "a8eaaef316df9b04c21a93dc10fcc27e"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.178774 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:23.179481 25198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.179706 25184 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 18:00:23.179826 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal
uuid: "b321187b0fd4475faf42154b4c2eda88"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "e60566880f156840b26c63fc55e2073e"
server_key_iv: "a8eaaef316df9b04c21a93dc10fcc27e"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.179939 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:23.198980 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:23.199275 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:23.199380 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:23.199570 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:23.200053 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:23.200093 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.200116 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:23.200157 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.206154 25184 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.66:33753
I20260928 18:00:23.206202 25311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.66:33753 every 8 connection(s)
I20260928 18:00:23.206542 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/data/info.pb
I20260928 18:00:23.212499 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.207301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.66:48103 (local address 127.5.46.126:35191)
0928 18:00:23.207395 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.207398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.207617 (+ 219us) server_negotiation.cc:407] Connection header received
0928 18:00:23.208257 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.208259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.208270 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.208345 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:23.209033 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.209528 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.210580 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.210895 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.210960 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.212199 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:23.212200 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:23.212200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:23.212205 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:23.212221 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:23.212225 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:23.212279 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.212395 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.212434 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260928 18:00:23.212795 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.207266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35191 (local address 127.5.46.66:48103)
0928 18:00:23.207489 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.207526 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.208109 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.208424 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.208427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.208435 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:23.208946 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.208950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.209659 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.209660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.210474 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.210477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.211001 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.211002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.211133 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.211974 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:23.211975 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:23.212015 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:23.212063 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:23.212288 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:23.212293 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:23.212296 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.212353 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.212447 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":123,"threads_started":1}
I20260928 18:00:23.213459 25312 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35191
I20260928 18:00:23.213583 25312 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:23.213806 25312 heartbeater.cc:511] Master 127.5.46.126:35191 requested a full tablet report, sending...
I20260928 18:00:23.214239 24993 ts_manager.cc:194] Registered new tserver with Master: b321187b0fd4475faf42154b4c2eda88 (127.5.46.66:33753)
I20260928 18:00:23.214495 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25184
I20260928 18:00:23.214614 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal/instance
I20260928 18:00:23.214687 24993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.66:48103
I20260928 18:00:23.214862 5305 external_mini_cluster.cc:1428] Setting key cc2f4ca2253f426a984649d67fc82d14
I20260928 18:00:23.215813 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.46.67:0
--local_ip_for_outbound_sockets=127.5.46.67
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--builtin_ntp_servers=127.5.46.84:45077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 18:00:23.299575 25315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:23.299839 25315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:23.299948 25315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:23.301460 25315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:23.301517 25315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:23.301544 25315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.46.67
I20260928 18:00:23.303004 25315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.46.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.46.67
--webserver_port=0
--tserver_master_addrs=127.5.46.126:35191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.46.67
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:23.303234 25315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:23.303509 25315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:23.308775 25320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:23.308806 25321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:23.308810 25323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:23.309056 25315 server_base.cc:1034] running on GCE node
I20260928 18:00:23.309290 25315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:23.309567 25315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:23.310760 25315 hybrid_clock.cc:648] HybridClock initialized: now 1790618423310743 us; error 59 us; skew 500 ppm
I20260928 18:00:23.312520 25315 webserver.cc:460] Webserver started at http://127.5.46.67:46851/ using document root <none> and password file <none>
I20260928 18:00:23.312805 25315 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:23.312880 25315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:23.313026 25315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:23.314170 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data/instance:
uuid: "8cfbea1d2b464a0f84175cba263857af"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "b4aabb1161b654f18934a3a9b9fc26f6"
server_key_iv: "ab87eb33131a7ba5b2bbe4fba5d100b3"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.314558 25315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance:
uuid: "8cfbea1d2b464a0f84175cba263857af"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "b4aabb1161b654f18934a3a9b9fc26f6"
server_key_iv: "ab87eb33131a7ba5b2bbe4fba5d100b3"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.316069 25315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 18:00:23.317091 25329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.317409 25315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:23.317562 25315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal
uuid: "8cfbea1d2b464a0f84175cba263857af"
format_stamp: "Formatted at 2026-09-28 18:00:23 on dist-test-slave-z2t3"
server_key: "b4aabb1161b654f18934a3a9b9fc26f6"
server_key_iv: "ab87eb33131a7ba5b2bbe4fba5d100b3"
server_key_version: "encryptionkey@0"
I20260928 18:00:23.317710 25315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:23.328956 25315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:23.329306 25315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:23.329465 25315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:23.329725 25315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 18:00:23.330072 25315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 18:00:23.330128 25315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.330170 25315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 18:00:23.330199 25315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:23.336457 25315 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.67:45079
I20260928 18:00:23.336508 25442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.67:45079 every 8 connection(s)
I20260928 18:00:23.336869 25315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/data/info.pb
I20260928 18:00:23.341388 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25315
I20260928 18:00:23.341526 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal/instance
I20260928 18:00:23.341758 5305 external_mini_cluster.cc:1428] Setting key 9e80913b4b9c7edba31e898393d60cdc
I20260928 18:00:23.342841 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.337631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:34497 (local address 127.5.46.126:35191)
0928 18:00:23.337692 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.337702 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.337946 (+ 244us) server_negotiation.cc:407] Connection header received
0928 18:00:23.338453 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.338454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.338464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.338531 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:23.339296 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.339831 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.340990 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.341305 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.341394 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.342481 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:23.342482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:23.342482 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:23.342487 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:23.342505 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:23.342509 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:23.342561 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.342670 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.342769 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 18:00:23.343191 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.337585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:35191 (local address 127.5.46.67:34497)
0928 18:00:23.337804 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.337847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.338364 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.338615 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.338619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.338628 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:23.339156 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.339176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.340012 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.340014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.340877 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.340880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.341407 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.341407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.341583 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.342343 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 18:00:23.342344 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:23.342387 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 18:00:23.342414 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:23.342581 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:23.342587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:23.342590 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.342666 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.342811 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260928 18:00:23.343680 25443 heartbeater.cc:347] Connected to a master server at 127.5.46.126:35191
I20260928 18:00:23.343775 25443 heartbeater.cc:464] Registering TS with master...
I20260928 18:00:23.344015 25443 heartbeater.cc:511] Master 127.5.46.126:35191 requested a full tablet report, sending...
I20260928 18:00:23.344467 24993 ts_manager.cc:194] Registered new tserver with Master: 8cfbea1d2b464a0f84175cba263857af (127.5.46.67:45079)
I20260928 18:00:23.344872 24993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.46.67:34497
I20260928 18:00:23.354097 5305 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 18:00:23.358830 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.355725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.5.46.126:35191)
0928 18:00:23.355793 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.355795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.356039 (+ 244us) server_negotiation.cc:407] Connection header received
0928 18:00:23.356068 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.356069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.356083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.356143 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:23.356384 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.356861 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.357579 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.357758 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.357809 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.358609 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:23.358609 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:23.358610 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:23.358614 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:23.358625 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:23.358629 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:23.358668 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.358744 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.358766 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 18:00:23.360832 24993 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 18:00:23.362569 24993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 18:00:23.368966 25452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.364972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33753 (local address 127.0.0.1:40256)
0928 18:00:23.365158 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.365204 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.365277 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.365490 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.365493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.365496 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:23.365569 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.365571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.366692 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.366693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.367670 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.367672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.368642 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.368643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.368725 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.368740 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.368815 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.368868 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260928 18:00:23.368966 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.365056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40256 (local address 127.5.46.66:33753)
0928 18:00:23.365241 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.365246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.365258 (+ 12us) server_negotiation.cc:407] Connection header received
0928 18:00:23.365371 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.365374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.365393 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.365487 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:23.365689 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.366595 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.367765 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.368564 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.368628 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.368724 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.368843 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.368881 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260928 18:00:23.369656 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.364975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.67:45079 (local address 127.0.0.1:48818)
0928 18:00:23.365451 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.365462 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.365519 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.366052 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.366053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.366055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:23.366091 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.366093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.367406 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.367407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.368486 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.368489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.369402 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.369403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.369462 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.369466 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.369533 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.369574 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":30,"thread_start_us":62,"threads_started":1}
I20260928 18:00:23.369668 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.365262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.5.46.67:45079)
0928 18:00:23.365849 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.365853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.365862 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:23.365898 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.365900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.365915 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.365986 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:23.366168 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.367298 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.368563 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.369333 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.369402 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.369488 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.369548 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.369586 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":93,"threads_started":1}
I20260928 18:00:23.369796 25246 tablet_service.cc:1467] Processing CreateTablet for tablet 52696ae43f2b43caa94689c5ca638a98 (DEFAULT_TABLE table=test-table [id=49b39d68fc9047f98175adc89391acd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:23.370175 25246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52696ae43f2b43caa94689c5ca638a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:23.370398 25377 tablet_service.cc:1467] Processing CreateTablet for tablet 52696ae43f2b43caa94689c5ca638a98 (DEFAULT_TABLE table=test-table [id=49b39d68fc9047f98175adc89391acd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:23.370477 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.365190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41617 (local address 127.0.0.1:59618)
0928 18:00:23.365621 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.365636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.365702 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.365940 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.365941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.365943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:23.365988 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.365990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.367405 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.367406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.368479 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.368482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.370144 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.370145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.370213 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.370218 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.370274 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.370320 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":21,"thread_start_us":55,"threads_started":1}
I20260928 18:00:23.370724 25377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52696ae43f2b43caa94689c5ca638a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:23.370859 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.365397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.5.46.65:41617)
0928 18:00:23.365573 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.365577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.365698 (+ 121us) server_negotiation.cc:407] Connection header received
0928 18:00:23.365768 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.365770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.365787 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.365870 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:23.366141 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.367300 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.368588 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.369649 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.370016 (+ 367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.370104 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.370659 (+ 555us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.370710 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260928 18:00:23.371510 25115 tablet_service.cc:1467] Processing CreateTablet for tablet 52696ae43f2b43caa94689c5ca638a98 (DEFAULT_TABLE table=test-table [id=49b39d68fc9047f98175adc89391acd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 18:00:23.371878 25115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52696ae43f2b43caa94689c5ca638a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:23.373613 25457 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Bootstrap starting.
I20260928 18:00:23.374183 25458 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Bootstrap starting.
I20260928 18:00:23.374418 25457 tablet_bootstrap.cc:654] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:23.374861 25458 tablet_bootstrap.cc:654] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:23.374892 25457 log.cc:824] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:23.375190 25458 log.cc:824] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:23.375581 25460 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: Bootstrap starting.
I20260928 18:00:23.375658 25457 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: No bootstrap required, opened a new log
I20260928 18:00:23.375725 25457 ts_tablet_manager.cc:1397] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:23.376012 25458 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: No bootstrap required, opened a new log
I20260928 18:00:23.376097 25458 ts_tablet_manager.cc:1397] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:23.376346 25460 tablet_bootstrap.cc:654] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:23.376719 25460 log.cc:824] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:23.377537 25460 tablet_bootstrap.cc:492] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: No bootstrap required, opened a new log
I20260928 18:00:23.377611 25460 ts_tablet_manager.cc:1397] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 18:00:23.378340 25458 raft_consensus.cc:348] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.378521 25458 raft_consensus.cc:374] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:23.378566 25458 raft_consensus.cc:729] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfbea1d2b464a0f84175cba263857af, State: Initialized, Role: FOLLOWER
I20260928 18:00:23.378563 25457 raft_consensus.cc:348] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.378736 25457 raft_consensus.cc:374] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:23.378720 25458 consensus_queue.cc:260] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.378782 25457 raft_consensus.cc:729] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b321187b0fd4475faf42154b4c2eda88, State: Initialized, Role: FOLLOWER
I20260928 18:00:23.378918 25457 consensus_queue.cc:260] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.379069 25458 ts_tablet_manager.cc:1428] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 18:00:23.379215 25443 heartbeater.cc:503] Master 127.5.46.126:35191 was elected leader, sending a full tablet report...
I20260928 18:00:23.379227 25457 ts_tablet_manager.cc:1428] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 18:00:23.379346 25312 heartbeater.cc:503] Master 127.5.46.126:35191 was elected leader, sending a full tablet report...
I20260928 18:00:23.380327 25460 raft_consensus.cc:348] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.380505 25460 raft_consensus.cc:374] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:23.380551 25460 raft_consensus.cc:729] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70afccbb4cff4855a5d9f1579ccbfcd1, State: Initialized, Role: FOLLOWER
I20260928 18:00:23.380692 25460 consensus_queue.cc:260] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.380920 25460 ts_tablet_manager.cc:1428] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 18:00:23.380993 25181 heartbeater.cc:503] Master 127.5.46.126:35191 was elected leader, sending a full tablet report...
I20260928 18:00:23.394107 25463 raft_consensus.cc:482] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 18:00:23.394271 25463 raft_consensus.cc:504] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.394716 25463 leader_election.cc:288] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b321187b0fd4475faf42154b4c2eda88 (127.5.46.66:33753), 70afccbb4cff4855a5d9f1579ccbfcd1 (127.5.46.65:41617)
I20260928 18:00:23.398940 25453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.395049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:52279 (local address 127.5.46.66:33753)
0928 18:00:23.395114 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.395117 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.395152 (+ 35us) server_negotiation.cc:407] Connection header received
0928 18:00:23.395183 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.395184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.395198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.395248 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:23.395439 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.396251 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.397923 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.398515 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.398605 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.398676 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.398818 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.398865 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 18:00:23.398949 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.395272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.46.67:58363 (local address 127.5.46.65:41617)
0928 18:00:23.395320 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.395321 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.395337 (+ 16us) server_negotiation.cc:407] Connection header received
0928 18:00:23.395365 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.395366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.395376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.395452 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 18:00:23.395569 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.396285 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.397923 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.398517 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.398605 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.398616 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.398845 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.398887 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 18:00:23.398941 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.394967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.66:33753 (local address 127.5.46.67:52279)
0928 18:00:23.395053 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.395066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.395111 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.395281 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.395282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.395284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:23.395364 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.395366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.396392 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.396393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.397789 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.397793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.398627 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.398628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.398709 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.398713 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.398786 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.398847 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":112}
I20260928 18:00:23.398962 25466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.395065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41617 (local address 127.5.46.67:58363)
0928 18:00:23.395266 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.395275 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:23.395315 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:23.395455 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:23.395455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:23.395456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 18:00:23.395503 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.395504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.396392 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.396393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.397789 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:23.397793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.398627 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.398628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:23.398783 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.398788 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 18:00:23.398839 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 18:00:23.398867 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":57,"thread_start_us":137,"threads_started":1}
I20260928 18:00:23.399359 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52696ae43f2b43caa94689c5ca638a98" candidate_uuid: "8cfbea1d2b464a0f84175cba263857af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" is_pre_election: true
I20260928 18:00:23.399394 25266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52696ae43f2b43caa94689c5ca638a98" candidate_uuid: "8cfbea1d2b464a0f84175cba263857af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b321187b0fd4475faf42154b4c2eda88" is_pre_election: true
I20260928 18:00:23.399551 25266 raft_consensus.cc:2454] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cfbea1d2b464a0f84175cba263857af in term 0.
I20260928 18:00:23.399552 25135 raft_consensus.cc:2454] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8cfbea1d2b464a0f84175cba263857af in term 0.
I20260928 18:00:23.399824 25333 leader_election.cc:302] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70afccbb4cff4855a5d9f1579ccbfcd1, 8cfbea1d2b464a0f84175cba263857af; no voters:
I20260928 18:00:23.400064 25463 raft_consensus.cc:2781] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 18:00:23.400127 25463 raft_consensus.cc:482] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 18:00:23.400151 25463 raft_consensus.cc:3037] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:23.400866 25463 raft_consensus.cc:504] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.401008 25463 leader_election.cc:288] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [CANDIDATE]: Term 1 election: Requested vote from peers b321187b0fd4475faf42154b4c2eda88 (127.5.46.66:33753), 70afccbb4cff4855a5d9f1579ccbfcd1 (127.5.46.65:41617)
I20260928 18:00:23.401194 25266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52696ae43f2b43caa94689c5ca638a98" candidate_uuid: "8cfbea1d2b464a0f84175cba263857af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b321187b0fd4475faf42154b4c2eda88"
I20260928 18:00:23.401199 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52696ae43f2b43caa94689c5ca638a98" candidate_uuid: "8cfbea1d2b464a0f84175cba263857af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1"
I20260928 18:00:23.401280 25266 raft_consensus.cc:3037] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:23.401280 25135 raft_consensus.cc:3037] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:23.402056 25266 raft_consensus.cc:2454] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cfbea1d2b464a0f84175cba263857af in term 1.
I20260928 18:00:23.402088 25135 raft_consensus.cc:2454] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8cfbea1d2b464a0f84175cba263857af in term 1.
I20260928 18:00:23.402254 25333 leader_election.cc:302] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70afccbb4cff4855a5d9f1579ccbfcd1, 8cfbea1d2b464a0f84175cba263857af; no voters:
I20260928 18:00:23.402376 25463 raft_consensus.cc:2781] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:23.402536 25463 raft_consensus.cc:686] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 1 LEADER]: Becoming Leader. State: Replica: 8cfbea1d2b464a0f84175cba263857af, State: Running, Role: LEADER
I20260928 18:00:23.402637 25463 consensus_queue.cc:237] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } }
I20260928 18:00:23.403512 24992 catalog_manager.cc:5495] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af reported cstate change: term changed from 0 to 1, leader changed from <none> to 8cfbea1d2b464a0f84175cba263857af (127.5.46.67). New cstate: current_term: 1 leader_uuid: "8cfbea1d2b464a0f84175cba263857af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8cfbea1d2b464a0f84175cba263857af" member_type: VOTER last_known_addr { host: "127.5.46.67" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 } health_report { overall_health: UNKNOWN } } }
I20260928 18:00:23.417866 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.415350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.5.46.67:45079)
0928 18:00:23.415421 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:23.415423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:23.415433 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:23.415480 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:23.415481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:23.415498 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:23.415577 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:23.415691 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.416272 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.417038 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:23.417233 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:23.417320 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:23.417503 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:23.417603 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:23.417699 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:23.417718 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 18:00:23.417736 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 18:00:23.419819 25135 raft_consensus.cc:1261] T 52696ae43f2b43caa94689c5ca638a98 P 70afccbb4cff4855a5d9f1579ccbfcd1 [term 1 FOLLOWER]: Refusing update from remote peer 8cfbea1d2b464a0f84175cba263857af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:23.419819 25266 raft_consensus.cc:1261] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 1 FOLLOWER]: Refusing update from remote peer 8cfbea1d2b464a0f84175cba263857af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:23.420327 25467 consensus_queue.cc:1035] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [LEADER]: Connected to new peer: Peer: permanent_uuid: "70afccbb4cff4855a5d9f1579ccbfcd1" member_type: VOTER last_known_addr { host: "127.5.46.65" port: 41617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:23.420483 25463 consensus_queue.cc:1035] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b321187b0fd4475faf42154b4c2eda88" member_type: VOTER last_known_addr { host: "127.5.46.66" port: 33753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:23.423313 25471 mvcc.cc:205] Tried to move back new op lower bound from 7334373062324596736 to 7334373062258098176. Current Snapshot: MvccSnapshot[applied={T|T < 7334373062324596736}]
I20260928 18:00:23.423635 25472 mvcc.cc:205] Tried to move back new op lower bound from 7334373062324596736 to 7334373062258098176. Current Snapshot: MvccSnapshot[applied={T|T < 7334373062324596736}]
I20260928 18:00:23.425545 24992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:23.425673 24992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 18:00:23.426462 24992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 305aa59edde54e1a8939365b7c75d770: Sending DeleteTablet for 3 replicas of tablet 52696ae43f2b43caa94689c5ca638a98
I20260928 18:00:23.426795 25246 tablet_service.cc:1514] Processing DeleteTablet for tablet 52696ae43f2b43caa94689c5ca638a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:23 UTC) from {username='slave'} at 127.0.0.1:40256
I20260928 18:00:23.426834 25115 tablet_service.cc:1514] Processing DeleteTablet for tablet 52696ae43f2b43caa94689c5ca638a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:23 UTC) from {username='slave'} at 127.0.0.1:59618
I20260928 18:00:23.426939 25377 tablet_service.cc:1514] Processing DeleteTablet for tablet 52696ae43f2b43caa94689c5ca638a98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 18:00:23 UTC) from {username='slave'} at 127.0.0.1:48818
I20260928 18:00:23.427176 25482 tablet_replica.cc:330] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: stopping tablet replica
I20260928 18:00:23.427211 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25053
I20260928 18:00:23.427356 25482 raft_consensus.cc:2229] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 1 LEADER]: Raft consensus shutting down.
I20260928 18:00:23.427549 25482 raft_consensus.cc:2258] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:23.428166 25482 ts_tablet_manager.cc:1905] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:23.429939 25482 ts_tablet_manager.cc:1918] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:23.430011 25482 log.cc:1196] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-2/wal/wals/52696ae43f2b43caa94689c5ca638a98
I20260928 18:00:23.430277 25482 ts_tablet_manager.cc:1939] T 52696ae43f2b43caa94689c5ca638a98 P 8cfbea1d2b464a0f84175cba263857af: Deleting consensus metadata
I20260928 18:00:23.430447 25480 tablet_replica.cc:330] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: stopping tablet replica
I20260928 18:00:23.430593 25480 raft_consensus.cc:2229] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 18:00:23.430680 24978 catalog_manager.cc:4841] TS 8cfbea1d2b464a0f84175cba263857af (127.5.46.67:45079): tablet 52696ae43f2b43caa94689c5ca638a98 (table test-table [id=49b39d68fc9047f98175adc89391acd5]) successfully deleted
I20260928 18:00:23.430719 25480 raft_consensus.cc:2258] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 18:00:23.431162 25480 ts_tablet_manager.cc:1905] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 18:00:23.433194 25480 ts_tablet_manager.cc:1918] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 18:00:23.433336 25480 log.cc:1196] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Deleting WAL directory at /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790618177091767-5305-0/minicluster-data/ts-1/wal/wals/52696ae43f2b43caa94689c5ca638a98
I20260928 18:00:23.433645 25480 ts_tablet_manager.cc:1939] T 52696ae43f2b43caa94689c5ca638a98 P b321187b0fd4475faf42154b4c2eda88: Deleting consensus metadata
I20260928 18:00:23.434207 24980 catalog_manager.cc:4841] TS b321187b0fd4475faf42154b4c2eda88 (127.5.46.66:33753): tablet 52696ae43f2b43caa94689c5ca638a98 (table test-table [id=49b39d68fc9047f98175adc89391acd5]) successfully deleted
W20260928 18:00:23.434993 24980 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.46.65:41617 (error 108)
I20260928 18:00:23.435218 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25184
I20260928 18:00:23.435813 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:23.435357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.65:41617 (local address 127.0.0.1:59634)
0928 18:00:23.435734 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:23.435774 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
W20260928 18:00:23.436455 24980 catalog_manager.cc:4584] TS 70afccbb4cff4855a5d9f1579ccbfcd1 (127.5.46.65:41617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52696ae43f2b43caa94689c5ca638a98: Network error: Client connection negotiation failed: client connection to 127.5.46.65:41617: connect: Connection refused (error 111)
I20260928 18:00:23.440281 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25315
I20260928 18:00:23.445078 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 24962
2026-09-28T18:00:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (698 ms)
[----------] 6 tests from EncryptionPolicyTest (8335 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:23 dist-test-slave-z2t3 krb5kdc[25486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:23 dist-test-slave-z2t3 krb5kdc[25486](info): set up 2 sockets
Sep 28 18:00:23 dist-test-slave-z2t3 krb5kdc[25486](info): commencing operation
krb5kdc: starting...
W20260928 18:00:24.492195 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:24.619730 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:24.702492 25499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:24.702728 25499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:24.702754 25499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:24.703990 25499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:24.704038 25499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:24.704052 25499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:24.704066 25499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:24.704078 25499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:24.705593 25499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:24.705807 25499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:24.705988 25499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:24.710956 25505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:24.710961 25507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:24.710960 25504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:24.711002 25499 server_base.cc:1034] running on GCE node
I20260928 18:00:24.711481 25499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:24.711833 25499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:24.713043 25499 hybrid_clock.cc:648] HybridClock initialized: now 1790618424713018 us; error 42 us; skew 500 ppm
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618424, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:24.715796 25499 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:24.716323 25499 webserver.cc:460] Webserver started at http://127.0.0.1:41655/ using document root <none> and password file <none>
I20260928 18:00:24.716594 25499 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:24.716665 25499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:24.716823 25499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:24.717922 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "a8bd4371b40c405396293de650a1a7e2"
format_stamp: "Formatted at 2026-09-28 18:00:24 on dist-test-slave-z2t3"
server_key: "930411786472339df33210905fdc32e7"
server_key_iv: "1c7e4eaac540edef71569f3b77c740f0"
server_key_version: "encryptionkey@0"
I20260928 18:00:24.718300 25499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "a8bd4371b40c405396293de650a1a7e2"
format_stamp: "Formatted at 2026-09-28 18:00:24 on dist-test-slave-z2t3"
server_key: "930411786472339df33210905fdc32e7"
server_key_iv: "1c7e4eaac540edef71569f3b77c740f0"
server_key_version: "encryptionkey@0"
I20260928 18:00:24.719732 25499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:24.720621 25514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:24.720914 25499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 18:00:24.721058 25499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "a8bd4371b40c405396293de650a1a7e2"
format_stamp: "Formatted at 2026-09-28 18:00:24 on dist-test-slave-z2t3"
server_key: "930411786472339df33210905fdc32e7"
server_key_iv: "1c7e4eaac540edef71569f3b77c740f0"
server_key_version: "encryptionkey@0"
I20260928 18:00:24.721153 25499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:24.738722 25499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:24.741268 25499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:24.741464 25499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:24.745872 25499 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38087
I20260928 18:00:24.745906 25566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38087 every 8 connection(s)
I20260928 18:00:24.746281 25499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:24.747473 25567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:24.750165 25567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: Bootstrap starting.
I20260928 18:00:24.750839 25567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:24.751164 25567 log.cc:824] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:24.752103 25567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: No bootstrap required, opened a new log
I20260928 18:00:24.753913 25567 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260928 18:00:24.754077 25567 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:24.754102 25567 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8bd4371b40c405396293de650a1a7e2, State: Initialized, Role: FOLLOWER
I20260928 18:00:24.754328 25567 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260928 18:00:24.754403 25567 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:24.754427 25567 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:24.754460 25567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:24.755162 25567 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260928 18:00:24.755281 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25499
I20260928 18:00:24.755318 25567 leader_election.cc:302] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8bd4371b40c405396293de650a1a7e2; no voters:
I20260928 18:00:24.755470 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:24.755597 25567 leader_election.cc:288] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:24.755628 25570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:24.755699 5305 external_mini_cluster.cc:1428] Setting key b92e3b524e5819b7d9183aba75f618cd
I20260928 18:00:24.755766 25570 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [term 1 LEADER]: Becoming Leader. State: Replica: a8bd4371b40c405396293de650a1a7e2, State: Running, Role: LEADER
I20260928 18:00:24.755885 25570 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260928 18:00:24.755921 25567 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:24.756678 25572 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8bd4371b40c405396293de650a1a7e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } } }
I20260928 18:00:24.756850 25572 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:24.757417 25571 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8bd4371b40c405396293de650a1a7e2. Latest consensus state: current_term: 1 leader_uuid: "a8bd4371b40c405396293de650a1a7e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8bd4371b40c405396293de650a1a7e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } } }
I20260928 18:00:24.757531 25571 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2 [sys.catalog]: This master's current role is: LEADER
W20260928 18:00:24.758235 25588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:24.758296 25588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:24.758344 25582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:24.758525 25582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:24.760234 25582 catalog_manager.cc:1347] Generated new cluster ID: af83c88cec0a4a25b352e7abbe1b5e80
I20260928 18:00:24.760293 25582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25486](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 18:00:24.768838 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:24.756303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34399 (local address 127.0.0.1:38087)
0928 18:00:24.756587 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:24.756597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:24.756631 (+ 34us) server_negotiation.cc:407] Connection header received
0928 18:00:24.756766 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:24.756773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:24.756839 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:24.757710 (+ 871us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:24.757846 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:24.759362 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:24.760549 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:24.761206 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:24.761326 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:24.764454 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:24.764458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:24.764459 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:24.764473 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:24.766735 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:24.767251 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:24.767254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:24.767255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:24.767390 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:24.767651 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:24.767652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:24.767652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:24.768022 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:24.768197 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:24.768332 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 18:00:24.768552 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":93,"threads_started":1}
I20260928 18:00:24.770246 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:24.779413 25582 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:24.780181 25582 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:24.792217 25582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8bd4371b40c405396293de650a1a7e2: Generated new TSK 0
I20260928 18:00:24.792497 25582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:24.801815 25575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:24.776244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52295 (local address 127.0.0.1:38087)
0928 18:00:24.776354 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:24.776357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:24.776629 (+ 272us) server_negotiation.cc:407] Connection header received
0928 18:00:24.776693 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:24.776694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:24.776707 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:24.776777 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:24.777587 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:24.778429 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:24.779683 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:24.780075 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:24.780154 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:24.798452 (+ 18298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:24.798454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:24.798455 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:24.798465 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:24.800361 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:24.800909 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:24.800910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:24.800911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:24.800978 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:24.801231 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:24.801232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:24.801232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:24.801383 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:24.801496 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:24.801642 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 18:00:24.801712 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 18:00:24.804119 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25499
W20260928 18:00:24.808727 25484 connection.cc:535] client connection to 127.0.0.1:38087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38087): Connection reset by peer (error 104)
2026-09-28T18:00:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25598](info): set up 2 sockets
Sep 28 18:00:24 dist-test-slave-z2t3 krb5kdc[25598](info): commencing operation
krb5kdc: starting...
W20260928 18:00:25.864042 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:25 dist-test-slave-z2t3 krb5kdc[25598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:25.992115 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35995
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:26.075524 25611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:26.075767 25611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:26.075793 25611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:26.077066 25611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:26.077121 25611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:26.077135 25611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:26.077148 25611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:26.077162 25611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:26.078749 25611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35995
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:26.079031 25611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:26.079279 25611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:26.084380 25619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:26.084461 25616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:26.084581 25611 server_base.cc:1034] running on GCE node
W20260928 18:00:26.084380 25617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:26.084865 25611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:26.085124 25611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:26.086279 25611 hybrid_clock.cc:648] HybridClock initialized: now 1790618426086269 us; error 32 us; skew 500 ppm
Sep 28 18:00:26 dist-test-slave-z2t3 krb5kdc[25598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618426, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:26.095945 25611 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:26.096485 25611 webserver.cc:460] Webserver started at http://127.0.0.1:41945/ using document root <none> and password file <none>
I20260928 18:00:26.096735 25611 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:26.096781 25611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:26.096954 25611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:26.098035 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "6511838eb32f44139b92c64c7aa3b348"
format_stamp: "Formatted at 2026-09-28 18:00:26 on dist-test-slave-z2t3"
server_key: "18bc44bba2eca064651d6da2aed6452b"
server_key_iv: "a522c557ee4604b5548fb5092c0643df"
server_key_version: "encryptionkey@0"
I20260928 18:00:26.098402 25611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "6511838eb32f44139b92c64c7aa3b348"
format_stamp: "Formatted at 2026-09-28 18:00:26 on dist-test-slave-z2t3"
server_key: "18bc44bba2eca064651d6da2aed6452b"
server_key_iv: "a522c557ee4604b5548fb5092c0643df"
server_key_version: "encryptionkey@0"
I20260928 18:00:26.099772 25611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:26.100631 25626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:26.100920 25611 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:26.101019 25611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "6511838eb32f44139b92c64c7aa3b348"
format_stamp: "Formatted at 2026-09-28 18:00:26 on dist-test-slave-z2t3"
server_key: "18bc44bba2eca064651d6da2aed6452b"
server_key_iv: "a522c557ee4604b5548fb5092c0643df"
server_key_version: "encryptionkey@0"
I20260928 18:00:26.101081 25611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:26.120510 25611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:26.122785 25611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:26.122913 25611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:26.127072 25611 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35995
I20260928 18:00:26.127170 25678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35995 every 8 connection(s)
I20260928 18:00:26.127525 25611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:26.128707 25679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:26.131278 25679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348: Bootstrap starting.
I20260928 18:00:26.132021 25679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:26.132354 25679 log.cc:824] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:26.133235 25679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348: No bootstrap required, opened a new log
I20260928 18:00:26.135039 25679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260928 18:00:26.135259 25679 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:26.135316 25679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6511838eb32f44139b92c64c7aa3b348, State: Initialized, Role: FOLLOWER
I20260928 18:00:26.135538 25679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260928 18:00:26.135654 25679 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:26.135700 25679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:26.135759 25679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:26.136437 25679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260928 18:00:26.136574 25679 leader_election.cc:302] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6511838eb32f44139b92c64c7aa3b348; no voters:
I20260928 18:00:26.136801 25679 leader_election.cc:288] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:26.136869 25682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:26.137100 25679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:26.137153 25682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [term 1 LEADER]: Becoming Leader. State: Replica: 6511838eb32f44139b92c64c7aa3b348, State: Running, Role: LEADER
I20260928 18:00:26.137269 25682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } }
I20260928 18:00:26.137648 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25611
I20260928 18:00:26.137765 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:26.137964 5305 external_mini_cluster.cc:1428] Setting key 32966e9188c68a4e4f37478884fc6f01
I20260928 18:00:26.138135 25684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6511838eb32f44139b92c64c7aa3b348. Latest consensus state: current_term: 1 leader_uuid: "6511838eb32f44139b92c64c7aa3b348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } } }
I20260928 18:00:26.138236 25684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:26.138283 25683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6511838eb32f44139b92c64c7aa3b348" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6511838eb32f44139b92c64c7aa3b348" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35995 } } }
I20260928 18:00:26.138350 25683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:26.138998 25691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:26.139653 25691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:26.141741 25691 catalog_manager.cc:1347] Generated new cluster ID: 66015cc440a2478e8ae21afa322e7e3e
I20260928 18:00:26.141810 25691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:26.144706 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:26.138545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46603 (local address 127.0.0.1:35995)
0928 18:00:26.139060 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:26.139072 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:26.139112 (+ 40us) server_negotiation.cc:407] Connection header received
0928 18:00:26.139270 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:26.139278 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:26.139346 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:26.139508 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:26.139804 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:26.141245 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:26.142370 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:26.142888 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:26.143027 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:26.143905 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:26.143910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:26.143910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:26.144140 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:26.144213 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:26.144228 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:26.144356 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:26.144447 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 18:00:26.144482 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":92,"threads_started":1}
I20260928 18:00:26.145900 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:26.148304 25691 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:26.149026 25691 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:26.151541 25696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:26.147962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38973 (local address 127.0.0.1:35995)
0928 18:00:26.148000 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:26.148002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:26.148228 (+ 226us) server_negotiation.cc:407] Connection header received
0928 18:00:26.148348 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:26.148349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:26.148363 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:26.148418 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:26.148641 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:26.149259 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:26.150066 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:26.150341 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:26.150425 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:26.151236 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:26.151237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:26.151238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:26.151244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:26.151261 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:26.151265 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:26.151347 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:26.151418 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 18:00:26.151432 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 18:00:26.158146 25691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6511838eb32f44139b92c64c7aa3b348: Generated new TSK 0
I20260928 18:00:26.158409 25691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:26.161422 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25611
W20260928 18:00:26.165078 25596 connection.cc:535] client connection to 127.0.0.1:35995 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35995): Connection reset by peer (error 104)
2026-09-28T18:00:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:26 dist-test-slave-z2t3 krb5kdc[25710](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:26 dist-test-slave-z2t3 krb5kdc[25710](info): set up 2 sockets
Sep 28 18:00:26 dist-test-slave-z2t3 krb5kdc[25710](info): commencing operation
krb5kdc: starting...
W20260928 18:00:27.224006 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:27 dist-test-slave-z2t3 krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:27.351707 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:27.437920 25723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:27.438318 25723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:27.438362 25723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:27.439618 25723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:27.439702 25723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:27.439729 25723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:27.439762 25723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:27.439790 25723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:27.441421 25723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39417
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:27.441723 25723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:27.441965 25723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:27.447089 25731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:27.447117 25729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:27.447089 25728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:27.447403 25723 server_base.cc:1034] running on GCE node
I20260928 18:00:27.447623 25723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:27.448001 25723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:27.449208 25723 hybrid_clock.cc:648] HybridClock initialized: now 1790618427449178 us; error 39 us; skew 500 ppm
Sep 28 18:00:27 dist-test-slave-z2t3 krb5kdc[25710](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618427, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:27.451830 25723 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:27.452407 25723 webserver.cc:460] Webserver started at http://127.0.0.1:35455/ using document root <none> and password file <none>
I20260928 18:00:27.452651 25723 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:27.452724 25723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:27.452885 25723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:27.453997 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "e071c6a1b9564c788005b8fde57525d7"
format_stamp: "Formatted at 2026-09-28 18:00:27 on dist-test-slave-z2t3"
server_key: "db49073ed4dfdabb0a898c36031d4527"
server_key_iv: "ded5d04a3a2726cb1f1555a32babc591"
server_key_version: "encryptionkey@0"
I20260928 18:00:27.454391 25723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "e071c6a1b9564c788005b8fde57525d7"
format_stamp: "Formatted at 2026-09-28 18:00:27 on dist-test-slave-z2t3"
server_key: "db49073ed4dfdabb0a898c36031d4527"
server_key_iv: "ded5d04a3a2726cb1f1555a32babc591"
server_key_version: "encryptionkey@0"
I20260928 18:00:27.455833 25723 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260928 18:00:27.456691 25738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:27.456890 25723 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:27.456992 25723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "e071c6a1b9564c788005b8fde57525d7"
format_stamp: "Formatted at 2026-09-28 18:00:27 on dist-test-slave-z2t3"
server_key: "db49073ed4dfdabb0a898c36031d4527"
server_key_iv: "ded5d04a3a2726cb1f1555a32babc591"
server_key_version: "encryptionkey@0"
I20260928 18:00:27.457080 25723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:27.460623 25723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:27.463100 25723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:27.463310 25723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:27.467550 25723 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39417
I20260928 18:00:27.467586 25790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39417 every 8 connection(s)
I20260928 18:00:27.468026 25723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:27.469045 25791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:27.471342 25791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7: Bootstrap starting.
I20260928 18:00:27.471969 25791 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:27.472229 25791 log.cc:824] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:27.472914 25791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7: No bootstrap required, opened a new log
I20260928 18:00:27.474457 25791 raft_consensus.cc:348] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } }
I20260928 18:00:27.474604 25791 raft_consensus.cc:374] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:27.474628 25791 raft_consensus.cc:729] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e071c6a1b9564c788005b8fde57525d7, State: Initialized, Role: FOLLOWER
I20260928 18:00:27.474779 25791 consensus_queue.cc:260] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } }
I20260928 18:00:27.474864 25791 raft_consensus.cc:388] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:27.474888 25791 raft_consensus.cc:482] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:27.474915 25791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:27.475442 25791 raft_consensus.cc:504] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } }
I20260928 18:00:27.475526 25791 leader_election.cc:302] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e071c6a1b9564c788005b8fde57525d7; no voters:
I20260928 18:00:27.475661 25791 leader_election.cc:288] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:27.475751 25794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:27.475951 25794 raft_consensus.cc:686] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [term 1 LEADER]: Becoming Leader. State: Replica: e071c6a1b9564c788005b8fde57525d7, State: Running, Role: LEADER
I20260928 18:00:27.475987 25791 sys_catalog.cc:564] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:27.476063 25794 consensus_queue.cc:237] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } }
I20260928 18:00:27.476491 25796 sys_catalog.cc:455] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e071c6a1b9564c788005b8fde57525d7. Latest consensus state: current_term: 1 leader_uuid: "e071c6a1b9564c788005b8fde57525d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } } }
I20260928 18:00:27.476581 25796 sys_catalog.cc:458] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:27.476723 25795 sys_catalog.cc:455] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e071c6a1b9564c788005b8fde57525d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e071c6a1b9564c788005b8fde57525d7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39417 } } }
I20260928 18:00:27.476846 25795 sys_catalog.cc:458] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:27.477006 25803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:27.477335 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25723
I20260928 18:00:27.477453 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:27.477576 25803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:27.477630 5305 external_mini_cluster.cc:1428] Setting key f1632d14fef5f09120a3a61c29376f0d
I20260928 18:00:27.479907 25803 catalog_manager.cc:1347] Generated new cluster ID: b6b4fa8a805548f3960e8a2a7b6fb1e2
I20260928 18:00:27.479978 25803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:27.480044 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:27.478296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39899 (local address 127.0.0.1:39417)
0928 18:00:27.478462 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:27.478472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:27.478503 (+ 31us) server_negotiation.cc:407] Connection header received
0928 18:00:27.478610 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:27.478615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:27.478639 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:27.478785 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:27.479502 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:27.479505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:27.479506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:27.479510 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:27.479555 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:27.479567 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:27.479640 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:27.479711 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 18:00:27.479753 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260928 18:00:27.480929 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:27.483700 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:27.482528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46255 (local address 127.0.0.1:39417)
0928 18:00:27.482565 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:27.482567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:27.482600 (+ 33us) server_negotiation.cc:407] Connection header received
0928 18:00:27.482623 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:27.482624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:27.482640 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:27.482757 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:27.483438 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:27.483439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:27.483440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:27.483444 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:27.483459 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:27.483464 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:27.483494 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:27.483601 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 18:00:27.483616 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 18:00:27.487560 25803 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:27.487754 25803 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:27.492516 25803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e071c6a1b9564c788005b8fde57525d7: Generated new TSK 0
I20260928 18:00:27.492726 25803 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:27.495393 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25723
2026-09-28T18:00:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:27 dist-test-slave-z2t3 krb5kdc[25822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:27 dist-test-slave-z2t3 krb5kdc[25822](info): set up 2 sockets
Sep 28 18:00:27 dist-test-slave-z2t3 krb5kdc[25822](info): commencing operation
krb5kdc: starting...
W20260928 18:00:28.536165 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:28.657029 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:28.741909 25835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:28.742129 25835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:28.742153 25835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:28.743408 25835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:28.743456 25835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:28.743471 25835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:28.743484 25835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:28.743496 25835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:28.745056 25835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:28.745324 25835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:28.745565 25835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:28.750818 25843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:28.750818 25840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:28.750885 25835 server_base.cc:1034] running on GCE node
W20260928 18:00:28.750849 25841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:28.751313 25835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:28.751663 25835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:28.752846 25835 hybrid_clock.cc:648] HybridClock initialized: now 1790618428752823 us; error 41 us; skew 500 ppm
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618428, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:28.755456 25835 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:28.755931 25835 webserver.cc:460] Webserver started at http://127.0.0.1:43315/ using document root <none> and password file <none>
I20260928 18:00:28.756176 25835 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:28.756228 25835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:28.756323 25835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:28.757352 25835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "213b0c8e172c4ce49c5634f77c1bbc4b"
format_stamp: "Formatted at 2026-09-28 18:00:28 on dist-test-slave-z2t3"
server_key: "ee95ed96fd08ea372622a4dc2212756d"
server_key_iv: "9f8a66265644affba174d0e33b508b17"
server_key_version: "encryptionkey@0"
I20260928 18:00:28.757652 25835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "213b0c8e172c4ce49c5634f77c1bbc4b"
format_stamp: "Formatted at 2026-09-28 18:00:28 on dist-test-slave-z2t3"
server_key: "ee95ed96fd08ea372622a4dc2212756d"
server_key_iv: "9f8a66265644affba174d0e33b508b17"
server_key_version: "encryptionkey@0"
I20260928 18:00:28.758963 25835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:28.759670 25850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:28.759902 25835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:28.759996 25835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "213b0c8e172c4ce49c5634f77c1bbc4b"
format_stamp: "Formatted at 2026-09-28 18:00:28 on dist-test-slave-z2t3"
server_key: "ee95ed96fd08ea372622a4dc2212756d"
server_key_iv: "9f8a66265644affba174d0e33b508b17"
server_key_version: "encryptionkey@0"
I20260928 18:00:28.760054 25835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:28.771143 25835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:28.773710 25835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:28.773845 25835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:28.777740 25835 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33733
I20260928 18:00:28.777806 25902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33733 every 8 connection(s)
I20260928 18:00:28.778168 25835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:28.779189 25903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:28.781600 25903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b: Bootstrap starting.
I20260928 18:00:28.782188 25903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:28.782430 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25835
I20260928 18:00:28.782529 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:28.782526 25903 log.cc:824] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:28.782657 5305 external_mini_cluster.cc:1428] Setting key c4bfc7bcd722c01d0c088ef608385f47
I20260928 18:00:28.783497 25903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b: No bootstrap required, opened a new log
I20260928 18:00:28.786073 25903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260928 18:00:28.786257 25903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:28.786293 25903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 213b0c8e172c4ce49c5634f77c1bbc4b, State: Initialized, Role: FOLLOWER
I20260928 18:00:28.786427 25903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260928 18:00:28.786513 25903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:28.786546 25903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:28.786593 25903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:28.787348 25903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260928 18:00:28.787469 25903 leader_election.cc:302] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 213b0c8e172c4ce49c5634f77c1bbc4b; no voters:
I20260928 18:00:28.787674 25903 leader_election.cc:288] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:28.787719 25908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:28.787943 25908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [term 1 LEADER]: Becoming Leader. State: Replica: 213b0c8e172c4ce49c5634f77c1bbc4b, State: Running, Role: LEADER
I20260928 18:00:28.787977 25903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:28.788111 25908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } }
I20260928 18:00:28.789120 25909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260928 18:00:28.789283 25909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:28.789577 25918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:28.789973 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 213b0c8e172c4ce49c5634f77c1bbc4b. Latest consensus state: current_term: 1 leader_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "213b0c8e172c4ce49c5634f77c1bbc4b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33733 } } }
I20260928 18:00:28.790110 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:28.790143 25918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 18:00:28.791949 25918 catalog_manager.cc:1347] Generated new cluster ID: 482ee1271ed84d908bf87dd5b5665ec8
I20260928 18:00:28.792001 25918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:28.795120 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:28.783140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50045 (local address 127.0.0.1:33733)
0928 18:00:28.783334 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:28.783343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:28.783375 (+ 32us) server_negotiation.cc:407] Connection header received
0928 18:00:28.783999 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:28.784009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:28.784075 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:28.784230 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:28.784858 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:28.786311 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:28.787433 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:28.788188 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:28.788334 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:28.791207 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:28.791211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:28.791211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:28.791223 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:28.793380 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:28.793802 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:28.793806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:28.793806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:28.793923 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:28.794133 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:28.794134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:28.794134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:28.794464 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:28.794613 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:28.794737 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 18:00:28.794896 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260928 18:00:28.796187 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:28.802687 25918 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:28.803395 25918 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:28.805730 25907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:28.797649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37471 (local address 127.0.0.1:33733)
0928 18:00:28.797687 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:28.797689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:28.797864 (+ 175us) server_negotiation.cc:407] Connection header received
0928 18:00:28.797898 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:28.797899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:28.797910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:28.797978 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:28.798574 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:28.799338 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:28.800384 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:28.800718 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:28.801117 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:28.802733 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:28.802734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:28.802735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:28.802742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:28.804507 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:28.804921 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:28.804922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:28.804922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:28.804988 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:28.805186 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:28.805186 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:28.805187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:28.805319 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:28.805427 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:28.805536 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 18:00:28.805631 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 18:00:28.808219 25918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 213b0c8e172c4ce49c5634f77c1bbc4b: Generated new TSK 0
I20260928 18:00:28.808436 25918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:28.814414 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25835
2026-09-28T18:00:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1318 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25934](info): set up 2 sockets
Sep 28 18:00:28 dist-test-slave-z2t3 krb5kdc[25934](info): commencing operation
krb5kdc: starting...
W20260928 18:00:29.892031 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:29 dist-test-slave-z2t3 krb5kdc[25934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618429, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:30.014472 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:30.096196 25947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:30.096415 25947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:30.096439 25947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:30.097600 25947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:30.097641 25947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:30.097656 25947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:30.097667 25947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:30.097679 25947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:30.099133 25947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:30.099342 25947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:30.099519 25947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 18:00:30.104602 25947 server_base.cc:1034] running on GCE node
W20260928 18:00:30.104597 25952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:30.104594 25953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:30.104805 25955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:30.105151 25947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:30.105455 25947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:30.106623 25947 hybrid_clock.cc:648] HybridClock initialized: now 1790618430106608 us; error 47 us; skew 500 ppm
Sep 28 18:00:30 dist-test-slave-z2t3 krb5kdc[25934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618430, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:30.109230 25947 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:30.109611 25947 webserver.cc:460] Webserver started at http://127.0.0.1:44511/ using document root <none> and password file <none>
I20260928 18:00:30.109859 25947 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:30.109907 25947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:30.109997 25947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:30.110993 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "40c89ea71fe5444e89d05c85ac39f013"
format_stamp: "Formatted at 2026-09-28 18:00:30 on dist-test-slave-z2t3"
server_key: "468f8189aced52a2f9e2afdd6b31a88a"
server_key_iv: "d602c2d435d276b36bb48c38b47ffbdd"
server_key_version: "encryptionkey@0"
I20260928 18:00:30.111346 25947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "40c89ea71fe5444e89d05c85ac39f013"
format_stamp: "Formatted at 2026-09-28 18:00:30 on dist-test-slave-z2t3"
server_key: "468f8189aced52a2f9e2afdd6b31a88a"
server_key_iv: "d602c2d435d276b36bb48c38b47ffbdd"
server_key_version: "encryptionkey@0"
I20260928 18:00:30.112788 25947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:30.113473 25962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:30.113667 25947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:30.113749 25947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "40c89ea71fe5444e89d05c85ac39f013"
format_stamp: "Formatted at 2026-09-28 18:00:30 on dist-test-slave-z2t3"
server_key: "468f8189aced52a2f9e2afdd6b31a88a"
server_key_iv: "d602c2d435d276b36bb48c38b47ffbdd"
server_key_version: "encryptionkey@0"
I20260928 18:00:30.113801 25947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:30.125444 25947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:30.127758 25947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:30.127939 25947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:30.131772 25947 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34279
I20260928 18:00:30.131834 26014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34279 every 8 connection(s)
I20260928 18:00:30.132247 25947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:30.133257 26015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:30.135484 26015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013: Bootstrap starting.
I20260928 18:00:30.136125 26015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:30.136423 26015 log.cc:824] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:30.137102 26015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013: No bootstrap required, opened a new log
I20260928 18:00:30.138641 26015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } }
I20260928 18:00:30.138818 26015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:30.138867 26015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c89ea71fe5444e89d05c85ac39f013, State: Initialized, Role: FOLLOWER
I20260928 18:00:30.139025 26015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } }
I20260928 18:00:30.139086 26015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:30.139107 26015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:30.139176 26015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:30.139658 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 25947
I20260928 18:00:30.139753 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:30.139743 26015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } }
I20260928 18:00:30.139849 26015 leader_election.cc:302] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c89ea71fe5444e89d05c85ac39f013; no voters:
I20260928 18:00:30.140034 5305 external_mini_cluster.cc:1428] Setting key 6ca5aba386c77888d3c885f7411b82a0
I20260928 18:00:30.140089 26015 leader_election.cc:288] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:30.140117 26018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:30.140280 26018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [term 1 LEADER]: Becoming Leader. State: Replica: 40c89ea71fe5444e89d05c85ac39f013, State: Running, Role: LEADER
I20260928 18:00:30.140395 26015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:30.140419 26018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } }
I20260928 18:00:30.141038 26018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c89ea71fe5444e89d05c85ac39f013. Latest consensus state: current_term: 1 leader_uuid: "40c89ea71fe5444e89d05c85ac39f013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } } }
I20260928 18:00:30.141047 26019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c89ea71fe5444e89d05c85ac39f013" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c89ea71fe5444e89d05c85ac39f013" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34279 } } }
I20260928 18:00:30.141204 26019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:30.141191 26018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:30.141439 26029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:30.141993 26029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:30.143668 26029 catalog_manager.cc:1347] Generated new cluster ID: 0a6470fcca754fc18c2b90987e2f821c
I20260928 18:00:30.143719 26029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:30.147325 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:30.140675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41495 (local address 127.0.0.1:34279)
0928 18:00:30.140830 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:30.140839 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:30.141842 (+ 1003us) server_negotiation.cc:407] Connection header received
0928 18:00:30.142024 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:30.142032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:30.142104 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:30.142246 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:30.142592 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:30.143917 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:30.144991 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:30.145604 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:30.145693 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:30.146498 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:30.146502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:30.146502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:30.146703 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:30.146740 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:30.146754 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:30.146802 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:30.147058 (+ 256us) server_negotiation.cc:299] Negotiation successful
0928 18:00:30.147122 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260928 18:00:30.148101 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:30.155989 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:30.152300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45661 (local address 127.0.0.1:34279)
0928 18:00:30.152339 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:30.152341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:30.152455 (+ 114us) server_negotiation.cc:407] Connection header received
0928 18:00:30.152518 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:30.152518 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:30.152530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:30.152595 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:30.152760 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:30.153479 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:30.154565 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:30.154871 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:30.154936 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:30.155683 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:30.155684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:30.155685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:30.155692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:30.155708 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:30.155712 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:30.155769 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:30.155840 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 18:00:30.155901 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:30.161156 26029 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:30.161688 26029 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:30.170276 26029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c89ea71fe5444e89d05c85ac39f013: Generated new TSK 0
I20260928 18:00:30.170490 26029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:30.175588 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 25947
2026-09-28T18:00:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:30 dist-test-slave-z2t3 krb5kdc[26046](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:30 dist-test-slave-z2t3 krb5kdc[26046](info): set up 2 sockets
Sep 28 18:00:30 dist-test-slave-z2t3 krb5kdc[26046](info): commencing operation
krb5kdc: starting...
W20260928 18:00:31.231992 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:31 dist-test-slave-z2t3 krb5kdc[26046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:31.352630 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46361
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46361
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:31.435061 26059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:31.435292 26059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:31.435318 26059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:31.436478 26059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:31.436518 26059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:31.436533 26059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:31.436546 26059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:31.436558 26059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:31.438071 26059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:31.438547 26059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:31.438836 26059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:31.444039 26065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:31.444113 26059 server_base.cc:1034] running on GCE node
W20260928 18:00:31.444049 26064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:31.444049 26067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:31.444566 26059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:31.444831 26059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:31.446005 26059 hybrid_clock.cc:648] HybridClock initialized: now 1790618431445998 us; error 44 us; skew 500 ppm
Sep 28 18:00:31 dist-test-slave-z2t3 krb5kdc[26046](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618431, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:31.461807 26059 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:31.462393 26059 webserver.cc:460] Webserver started at http://127.0.0.1:40415/ using document root <none> and password file <none>
I20260928 18:00:31.462668 26059 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:31.462742 26059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:31.462904 26059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:31.464095 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "c503ad27c28d40d4a669b63acc04c2af"
format_stamp: "Formatted at 2026-09-28 18:00:31 on dist-test-slave-z2t3"
server_key: "76b8902420c74910a68c5ea852d79514"
server_key_iv: "13913ee58c5e5b150d493ffac97b3cdc"
server_key_version: "encryptionkey@0"
I20260928 18:00:31.464471 26059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "c503ad27c28d40d4a669b63acc04c2af"
format_stamp: "Formatted at 2026-09-28 18:00:31 on dist-test-slave-z2t3"
server_key: "76b8902420c74910a68c5ea852d79514"
server_key_iv: "13913ee58c5e5b150d493ffac97b3cdc"
server_key_version: "encryptionkey@0"
I20260928 18:00:31.465929 26059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:31.466760 26074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:31.467005 26059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:31.467132 26059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "c503ad27c28d40d4a669b63acc04c2af"
format_stamp: "Formatted at 2026-09-28 18:00:31 on dist-test-slave-z2t3"
server_key: "76b8902420c74910a68c5ea852d79514"
server_key_iv: "13913ee58c5e5b150d493ffac97b3cdc"
server_key_version: "encryptionkey@0"
I20260928 18:00:31.467221 26059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:31.470863 26059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:31.473244 26059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:31.473421 26059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:31.477833 26059 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46361
I20260928 18:00:31.477877 26126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46361 every 8 connection(s)
I20260928 18:00:31.478245 26059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:31.479264 26127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:31.481717 26127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af: Bootstrap starting.
I20260928 18:00:31.482285 26127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:31.482556 26127 log.cc:824] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:31.483273 26127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af: No bootstrap required, opened a new log
I20260928 18:00:31.485008 26127 raft_consensus.cc:348] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } }
I20260928 18:00:31.485173 26127 raft_consensus.cc:374] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:31.485200 26127 raft_consensus.cc:729] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c503ad27c28d40d4a669b63acc04c2af, State: Initialized, Role: FOLLOWER
I20260928 18:00:31.485335 26127 consensus_queue.cc:260] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } }
I20260928 18:00:31.485394 26127 raft_consensus.cc:388] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:31.485419 26127 raft_consensus.cc:482] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:31.485456 26127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:31.485997 26127 raft_consensus.cc:504] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } }
I20260928 18:00:31.486088 26127 leader_election.cc:302] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c503ad27c28d40d4a669b63acc04c2af; no voters:
I20260928 18:00:31.486238 26127 leader_election.cc:288] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:31.486331 26130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:31.486500 26130 raft_consensus.cc:686] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [term 1 LEADER]: Becoming Leader. State: Replica: c503ad27c28d40d4a669b63acc04c2af, State: Running, Role: LEADER
I20260928 18:00:31.486531 26127 sys_catalog.cc:564] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:31.486591 26130 consensus_queue.cc:237] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } }
I20260928 18:00:31.487102 26131 sys_catalog.cc:455] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c503ad27c28d40d4a669b63acc04c2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } } }
I20260928 18:00:31.487130 26132 sys_catalog.cc:455] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [sys.catalog]: SysCatalogTable state changed. Reason: New leader c503ad27c28d40d4a669b63acc04c2af. Latest consensus state: current_term: 1 leader_uuid: "c503ad27c28d40d4a669b63acc04c2af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c503ad27c28d40d4a669b63acc04c2af" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46361 } } }
I20260928 18:00:31.487210 26131 sys_catalog.cc:458] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:31.487214 26132 sys_catalog.cc:458] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:31.487654 26134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:31.488219 26134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:31.488415 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26059
I20260928 18:00:31.488556 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:31.488776 5305 external_mini_cluster.cc:1428] Setting key 5c92ba0e0aed633a8ca6748278fdbf3e
I20260928 18:00:31.489850 26134 catalog_manager.cc:1347] Generated new cluster ID: 3cbb7f8e895d41fab41def8baf033c52
I20260928 18:00:31.489897 26134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:31.491387 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:31.489516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53741 (local address 127.0.0.1:46361)
0928 18:00:31.489742 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:31.489752 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:31.489794 (+ 42us) server_negotiation.cc:407] Connection header received
0928 18:00:31.489889 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:31.489895 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:31.489919 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:31.490072 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:31.490900 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:31.490904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:31.490904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:31.490909 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:31.490961 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:31.490974 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:31.491054 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:31.491135 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 18:00:31.491180 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":134,"threads_started":1}
I20260928 18:00:31.492583 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:31.501272 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:31.496212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44329 (local address 127.0.0.1:46361)
0928 18:00:31.496253 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:31.496255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:31.500109 (+ 3854us) server_negotiation.cc:407] Connection header received
0928 18:00:31.500205 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:31.500206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:31.500213 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:31.500293 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:31.501019 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:31.501021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:31.501021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:31.501026 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:31.501042 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:31.501046 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:31.501098 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:31.501151 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 18:00:31.501168 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 18:00:31.513756 26134 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:31.513953 26134 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:31.521431 26134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c503ad27c28d40d4a669b63acc04c2af: Generated new TSK 0
I20260928 18:00:31.521658 26134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:31.528115 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26059
2026-09-28T18:00:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:31 dist-test-slave-z2t3 krb5kdc[26158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:31 dist-test-slave-z2t3 krb5kdc[26158](info): set up 2 sockets
Sep 28 18:00:31 dist-test-slave-z2t3 krb5kdc[26158](info): commencing operation
krb5kdc: starting...
W20260928 18:00:32.608141 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:32.733166 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:32.817776 26171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:32.818009 26171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:32.818037 26171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:32.819209 26171 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:32.819259 26171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:32.819276 26171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:32.819291 26171 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:32.819305 26171 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:32.820832 26171 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:32.821065 26171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:32.821271 26171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:32.826472 26177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:32.826467 26179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:32.826464 26176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:32.826879 26171 server_base.cc:1034] running on GCE node
I20260928 18:00:32.827102 26171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:32.827378 26171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:32.828572 26171 hybrid_clock.cc:648] HybridClock initialized: now 1790618432828556 us; error 41 us; skew 500 ppm
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618432, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:32.831287 26171 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:32.831800 26171 webserver.cc:460] Webserver started at http://127.0.0.1:43347/ using document root <none> and password file <none>
I20260928 18:00:32.832104 26171 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:32.832178 26171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:32.832329 26171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:32.833478 26171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "b838d6aa83234c9c8cc220105a171f5b"
format_stamp: "Formatted at 2026-09-28 18:00:32 on dist-test-slave-z2t3"
server_key: "0cbe81d074375ad61d38e160c11a9757"
server_key_iv: "9096e7b3cbefeb53f4365ee54ba311d3"
server_key_version: "encryptionkey@0"
I20260928 18:00:32.833868 26171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "b838d6aa83234c9c8cc220105a171f5b"
format_stamp: "Formatted at 2026-09-28 18:00:32 on dist-test-slave-z2t3"
server_key: "0cbe81d074375ad61d38e160c11a9757"
server_key_iv: "9096e7b3cbefeb53f4365ee54ba311d3"
server_key_version: "encryptionkey@0"
I20260928 18:00:32.835286 26171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:32.836200 26186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:32.836382 26171 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:32.836499 26171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "b838d6aa83234c9c8cc220105a171f5b"
format_stamp: "Formatted at 2026-09-28 18:00:32 on dist-test-slave-z2t3"
server_key: "0cbe81d074375ad61d38e160c11a9757"
server_key_iv: "9096e7b3cbefeb53f4365ee54ba311d3"
server_key_version: "encryptionkey@0"
I20260928 18:00:32.836582 26171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:32.851636 26171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:32.853688 26171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:32.853870 26171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:32.857945 26171 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33463
I20260928 18:00:32.857980 26238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33463 every 8 connection(s)
I20260928 18:00:32.858354 26171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:32.859323 26239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:32.861590 26239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b: Bootstrap starting.
I20260928 18:00:32.862174 26239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:32.862473 26239 log.cc:824] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:32.863150 26239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b: No bootstrap required, opened a new log
I20260928 18:00:32.864782 26239 raft_consensus.cc:348] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } }
I20260928 18:00:32.864930 26239 raft_consensus.cc:374] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:32.864953 26239 raft_consensus.cc:729] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b838d6aa83234c9c8cc220105a171f5b, State: Initialized, Role: FOLLOWER
I20260928 18:00:32.865113 26239 consensus_queue.cc:260] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } }
I20260928 18:00:32.865175 26239 raft_consensus.cc:388] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:32.865226 26239 raft_consensus.cc:482] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:32.865278 26239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:32.865811 26239 raft_consensus.cc:504] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } }
I20260928 18:00:32.865922 26239 leader_election.cc:302] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b838d6aa83234c9c8cc220105a171f5b; no voters:
I20260928 18:00:32.866113 26239 leader_election.cc:288] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:32.866181 26242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:32.866355 26239 sys_catalog.cc:564] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:32.866361 26242 raft_consensus.cc:686] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [term 1 LEADER]: Becoming Leader. State: Replica: b838d6aa83234c9c8cc220105a171f5b, State: Running, Role: LEADER
I20260928 18:00:32.866503 26242 consensus_queue.cc:237] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } }
I20260928 18:00:32.866887 26243 sys_catalog.cc:455] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b838d6aa83234c9c8cc220105a171f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } } }
I20260928 18:00:32.866905 26244 sys_catalog.cc:455] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b838d6aa83234c9c8cc220105a171f5b. Latest consensus state: current_term: 1 leader_uuid: "b838d6aa83234c9c8cc220105a171f5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b838d6aa83234c9c8cc220105a171f5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33463 } } }
I20260928 18:00:32.867050 26243 sys_catalog.cc:458] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:32.867141 26244 sys_catalog.cc:458] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:32.867455 26249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:32.868009 26249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:32.868542 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26171
I20260928 18:00:32.868673 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:32.868882 5305 external_mini_cluster.cc:1428] Setting key 2694abfa5e1d70fc3712cb4aeb30bd7d
I20260928 18:00:32.869942 26249 catalog_manager.cc:1347] Generated new cluster ID: 7fa350fc14314ad598e66394bf026de2
I20260928 18:00:32.869995 26249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:32.880841 26249 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:32.883208 26249 catalog_manager.cc:1504] Loading token signing keys...
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 18:00:32.890702 26249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b838d6aa83234c9c8cc220105a171f5b: Generated new TSK 0
I20260928 18:00:32.890898 26249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:32.894290 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:32.869373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:33093 (local address 127.0.0.1:33463)
0928 18:00:32.869552 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:32.869562 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:32.869746 (+ 184us) server_negotiation.cc:407] Connection header received
0928 18:00:32.877471 (+ 7725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:32.877482 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:32.877570 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:32.877748 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:32.878720 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:32.881492 (+ 2772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:32.882639 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:32.883284 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:32.883415 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:32.890526 (+ 7111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:32.890531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:32.890531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:32.890546 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:32.892411 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:32.892801 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:32.892805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:32.892805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:32.892908 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:32.893147 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:32.893148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:32.893148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:32.893379 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:32.893968 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:32.894037 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 18:00:32.894115 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260928 18:00:32.895303 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:32.903257 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:32.896712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:47325 (local address 127.0.0.1:33463)
0928 18:00:32.896763 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:32.896765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:32.896867 (+ 102us) server_negotiation.cc:407] Connection header received
0928 18:00:32.896925 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:32.896925 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:32.896934 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:32.896976 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:32.897579 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:32.898062 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:32.899116 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:32.899324 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:32.899387 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:32.900732 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:32.900733 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:32.900733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:32.900739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:32.902111 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:32.902559 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:32.902560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:32.902560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:32.902609 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:32.902833 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:32.902833 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:32.902834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:32.902930 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:32.903013 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:32.903150 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 18:00:32.903197 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 18:00:32.905766 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26171
W20260928 18:00:32.910163 26156 connection.cc:535] client connection to 127.0.0.1:33463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33463): Connection reset by peer (error 104)
2026-09-28T18:00:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26270](info): set up 2 sockets
Sep 28 18:00:32 dist-test-slave-z2t3 krb5kdc[26270](info): commencing operation
krb5kdc: starting...
W20260928 18:00:33.956148 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:34 dist-test-slave-z2t3 krb5kdc[26270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:34.092219 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:34.173782 26283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:34.174002 26283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:34.174026 26283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:34.175134 26283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:34.175175 26283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:34.175189 26283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:34.175202 26283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:34.175215 26283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:34.176710 26283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:34.176951 26283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:34.177132 26283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:34.182336 26289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:34.182461 26291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:34.182349 26288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:34.182578 26283 server_base.cc:1034] running on GCE node
I20260928 18:00:34.182893 26283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:34.183188 26283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:34.184381 26283 hybrid_clock.cc:648] HybridClock initialized: now 1790618434184350 us; error 46 us; skew 500 ppm
Sep 28 18:00:34 dist-test-slave-z2t3 krb5kdc[26270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:34.187121 26283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:34.187603 26283 webserver.cc:460] Webserver started at http://127.0.0.1:41407/ using document root <none> and password file <none>
I20260928 18:00:34.187903 26283 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:34.187984 26283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:34.188148 26283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:34.189265 26283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "220cf0f88ccf469280c7bd7d13524677"
format_stamp: "Formatted at 2026-09-28 18:00:34 on dist-test-slave-z2t3"
server_key: "66d8e502f064a6717284934415c192a0"
server_key_iv: "839fbfd3a3e3b4e1e2f113dc322a3c3d"
server_key_version: "encryptionkey@0"
I20260928 18:00:34.189651 26283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "220cf0f88ccf469280c7bd7d13524677"
format_stamp: "Formatted at 2026-09-28 18:00:34 on dist-test-slave-z2t3"
server_key: "66d8e502f064a6717284934415c192a0"
server_key_iv: "839fbfd3a3e3b4e1e2f113dc322a3c3d"
server_key_version: "encryptionkey@0"
I20260928 18:00:34.191080 26283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:34.191906 26298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:34.192097 26283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:34.192203 26283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "220cf0f88ccf469280c7bd7d13524677"
format_stamp: "Formatted at 2026-09-28 18:00:34 on dist-test-slave-z2t3"
server_key: "66d8e502f064a6717284934415c192a0"
server_key_iv: "839fbfd3a3e3b4e1e2f113dc322a3c3d"
server_key_version: "encryptionkey@0"
I20260928 18:00:34.192291 26283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:34.205185 26283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:34.207499 26283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:34.207682 26283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:34.211807 26283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37353
I20260928 18:00:34.211886 26350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37353 every 8 connection(s)
I20260928 18:00:34.212330 26283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:34.213408 26351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:34.215763 26351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677: Bootstrap starting.
I20260928 18:00:34.216441 26351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:34.216717 26351 log.cc:824] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:34.217365 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26283
I20260928 18:00:34.217455 26351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677: No bootstrap required, opened a new log
I20260928 18:00:34.217542 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:34.217736 5305 external_mini_cluster.cc:1428] Setting key 4cf2cf28da4e8c5b58aeb96e3febb88a
I20260928 18:00:34.219457 26351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260928 18:00:34.219641 26351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:34.219693 26351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220cf0f88ccf469280c7bd7d13524677, State: Initialized, Role: FOLLOWER
I20260928 18:00:34.219930 26351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260928 18:00:34.220052 26351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:34.220101 26351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:34.220175 26351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:34.220794 26351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260928 18:00:34.220919 26351 leader_election.cc:302] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 220cf0f88ccf469280c7bd7d13524677; no voters:
I20260928 18:00:34.221105 26351 leader_election.cc:288] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:34.221149 26356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:34.221320 26356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [term 1 LEADER]: Becoming Leader. State: Replica: 220cf0f88ccf469280c7bd7d13524677, State: Running, Role: LEADER
I20260928 18:00:34.221396 26351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:34.221454 26356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260928 18:00:34.221858 26356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220cf0f88ccf469280c7bd7d13524677. Latest consensus state: current_term: 1 leader_uuid: "220cf0f88ccf469280c7bd7d13524677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } } }
I20260928 18:00:34.221976 26356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:34.222141 26357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "220cf0f88ccf469280c7bd7d13524677" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220cf0f88ccf469280c7bd7d13524677" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } } }
I20260928 18:00:34.222244 26357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:34.222434 26364 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:34.223208 26364 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:34.225049 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:34.218360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:44007 (local address 127.0.0.1:37353)
0928 18:00:34.218548 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:34.218557 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:34.218586 (+ 29us) server_negotiation.cc:407] Connection header received
0928 18:00:34.219430 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:34.219438 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:34.219495 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:34.219622 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:34.219939 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:34.221424 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:34.222589 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:34.223210 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:34.223705 (+ 495us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:34.224235 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:34.224239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:34.224240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:34.224476 (+ 236us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:34.224520 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:34.224536 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:34.224634 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:34.224795 (+ 161us) server_negotiation.cc:299] Negotiation successful
0928 18:00:34.224827 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260928 18:00:34.225373 26364 catalog_manager.cc:1347] Generated new cluster ID: badaea70fce24d52aa8306049399883b
I20260928 18:00:34.225423 26364 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:34.226046 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:34.235790 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:34.228203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:37839 (local address 127.0.0.1:37353)
0928 18:00:34.228242 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:34.228244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:34.232093 (+ 3849us) server_negotiation.cc:407] Connection header received
0928 18:00:34.232161 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:34.232162 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:34.232178 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:34.232259 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:34.232421 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:34.233193 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:34.234241 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:34.234614 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:34.234702 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:34.235500 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:34.235501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:34.235502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:34.235508 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:34.235526 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:34.235531 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:34.235596 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:34.235676 (+ 80us) server_negotiation.cc:299] Negotiation successful
0928 18:00:34.235692 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 18:00:34.262226 26364 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:34.262832 26364 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:34.269506 26364 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 220cf0f88ccf469280c7bd7d13524677: Generated new TSK 0
I20260928 18:00:34.269748 26364 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:34.280086 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26283
W20260928 18:00:34.284821 26268 connection.cc:535] client connection to 127.0.0.1:37353 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37353): Connection reset by peer (error 104)
2026-09-28T18:00:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:34 dist-test-slave-z2t3 krb5kdc[26382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:34 dist-test-slave-z2t3 krb5kdc[26382](info): set up 2 sockets
Sep 28 18:00:34 dist-test-slave-z2t3 krb5kdc[26382](info): commencing operation
krb5kdc: starting...
W20260928 18:00:35.316144 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:35 dist-test-slave-z2t3 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:35.443161 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:35.525167 26395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:35.525449 26395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:35.525489 26395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:35.526685 26395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:35.526762 26395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:35.526784 26395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:35.526805 26395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:35.526830 26395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:35.528407 26395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:35.528704 26395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:35.528950 26395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:35.534170 26400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:35.534170 26403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:35.534188 26401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:35.534574 26395 server_base.cc:1034] running on GCE node
I20260928 18:00:35.534845 26395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:35.535125 26395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:35.536265 26395 hybrid_clock.cc:648] HybridClock initialized: now 1790618435536252 us; error 32 us; skew 500 ppm
Sep 28 18:00:35 dist-test-slave-z2t3 krb5kdc[26382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618435, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:35.538887 26395 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:35.539389 26395 webserver.cc:460] Webserver started at http://127.0.0.1:35487/ using document root <none> and password file <none>
I20260928 18:00:35.539637 26395 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:35.539685 26395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:35.539876 26395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:35.540993 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "0c7f42ef9cf3444a9d64634decebd1d2"
format_stamp: "Formatted at 2026-09-28 18:00:35 on dist-test-slave-z2t3"
server_key: "0fab0dc0c8b8ac04b1e5ecf3c60fefb6"
server_key_iv: "3be3ea1de74116f41fffcbdb382393df"
server_key_version: "encryptionkey@0"
I20260928 18:00:35.541373 26395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "0c7f42ef9cf3444a9d64634decebd1d2"
format_stamp: "Formatted at 2026-09-28 18:00:35 on dist-test-slave-z2t3"
server_key: "0fab0dc0c8b8ac04b1e5ecf3c60fefb6"
server_key_iv: "3be3ea1de74116f41fffcbdb382393df"
server_key_version: "encryptionkey@0"
I20260928 18:00:35.542821 26395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:35.543697 26410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:35.543967 26395 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:35.544077 26395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "0c7f42ef9cf3444a9d64634decebd1d2"
format_stamp: "Formatted at 2026-09-28 18:00:35 on dist-test-slave-z2t3"
server_key: "0fab0dc0c8b8ac04b1e5ecf3c60fefb6"
server_key_iv: "3be3ea1de74116f41fffcbdb382393df"
server_key_version: "encryptionkey@0"
I20260928 18:00:35.544170 26395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:35.547636 26395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:35.550098 26395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:35.550280 26395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:35.554253 26395 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39757
I20260928 18:00:35.554284 26462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39757 every 8 connection(s)
I20260928 18:00:35.554632 26395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:35.555614 26463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:35.557956 26463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2: Bootstrap starting.
I20260928 18:00:35.558209 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26395
I20260928 18:00:35.558342 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:35.558547 5305 external_mini_cluster.cc:1428] Setting key 258127eae292862e9bcfc6d9ec25c59c
I20260928 18:00:35.558715 26463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:35.559052 26463 log.cc:824] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:35.559765 26463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2: No bootstrap required, opened a new log
I20260928 18:00:35.561259 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:35.559090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:60001 (local address 127.0.0.1:39757)
0928 18:00:35.559314 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:35.559323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:35.559355 (+ 32us) server_negotiation.cc:407] Connection header received
0928 18:00:35.559823 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:35.559832 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:35.559915 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:35.560076 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:35.560837 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:35.560841 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:35.560841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:35.560845 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:35.560895 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:35.560905 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:35.560995 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:35.561034 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 18:00:35.561077 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":84,"threads_started":1}
I20260928 18:00:35.562300 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:35.562165 26463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260928 18:00:35.562389 26463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:35.562443 26463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7f42ef9cf3444a9d64634decebd1d2, State: Initialized, Role: FOLLOWER
I20260928 18:00:35.562562 26463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260928 18:00:35.562668 26463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:35.562712 26463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:35.562760 26463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:35.563329 26463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260928 18:00:35.563433 26463 leader_election.cc:302] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c7f42ef9cf3444a9d64634decebd1d2; no voters:
I20260928 18:00:35.563609 26463 leader_election.cc:288] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:35.563675 26472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:35.563868 26472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [term 1 LEADER]: Becoming Leader. State: Replica: 0c7f42ef9cf3444a9d64634decebd1d2, State: Running, Role: LEADER
I20260928 18:00:35.563970 26472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } }
I20260928 18:00:35.564013 26463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:35.564296 26474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c7f42ef9cf3444a9d64634decebd1d2. Latest consensus state: current_term: 1 leader_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } } }
I20260928 18:00:35.564380 26474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:35.564530 26473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c7f42ef9cf3444a9d64634decebd1d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39757 } } }
I20260928 18:00:35.564651 26473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:35.565088 26478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:35.565385 26466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:35.563741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:56155 (local address 127.0.0.1:39757)
0928 18:00:35.563779 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:35.563781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:35.564120 (+ 339us) server_negotiation.cc:407] Connection header received
0928 18:00:35.564187 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:35.564188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:35.564194 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:35.564257 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:35.565075 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:35.565076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:35.565076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:35.565081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:35.565097 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:35.565102 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:35.565161 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:35.565312 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 18:00:35.565325 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:35.565649 26478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:35.567239 26478 catalog_manager.cc:1347] Generated new cluster ID: 90a3ddb7d14c4c71a1811c31ae35bde7
I20260928 18:00:35.567305 26478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:35.583427 26478 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:35.583632 26478 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:35.591347 26478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c7f42ef9cf3444a9d64634decebd1d2: Generated new TSK 0
I20260928 18:00:35.591572 26478 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:35.597090 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26395
2026-09-28T18:00:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:35 dist-test-slave-z2t3 krb5kdc[26494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:35 dist-test-slave-z2t3 krb5kdc[26494](info): set up 2 sockets
Sep 28 18:00:35 dist-test-slave-z2t3 krb5kdc[26494](info): commencing operation
krb5kdc: starting...
W20260928 18:00:36.660135 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:36 dist-test-slave-z2t3 krb5kdc[26494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:36.786707 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45955
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:36.868086 26507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:36.868307 26507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:36.868332 26507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:36.869447 26507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:36.869484 26507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:36.869498 26507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:36.869510 26507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:36.869522 26507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:36.870930 26507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:36.871130 26507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:36.871315 26507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:36.876242 26513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:36.876237 26512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:36.876313 26507 server_base.cc:1034] running on GCE node
W20260928 18:00:36.876242 26515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:36.876806 26507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:36.877105 26507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:36.878275 26507 hybrid_clock.cc:648] HybridClock initialized: now 1790618436878262 us; error 30 us; skew 500 ppm
Sep 28 18:00:36 dist-test-slave-z2t3 krb5kdc[26494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618436, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:36.885092 26507 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:36.885522 26507 webserver.cc:460] Webserver started at http://127.0.0.1:42043/ using document root <none> and password file <none>
I20260928 18:00:36.885733 26507 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:36.885774 26507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:36.885870 26507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:36.886901 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "c96ca95d807c4cff9818e178eef311c0"
format_stamp: "Formatted at 2026-09-28 18:00:36 on dist-test-slave-z2t3"
server_key: "35b35c4b3ad163451eec7e527a508e37"
server_key_iv: "ce7b9d31a9d2e825286f0f54bb936ca5"
server_key_version: "encryptionkey@0"
I20260928 18:00:36.887210 26507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "c96ca95d807c4cff9818e178eef311c0"
format_stamp: "Formatted at 2026-09-28 18:00:36 on dist-test-slave-z2t3"
server_key: "35b35c4b3ad163451eec7e527a508e37"
server_key_iv: "ce7b9d31a9d2e825286f0f54bb936ca5"
server_key_version: "encryptionkey@0"
I20260928 18:00:36.888581 26507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:36.889500 26522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:36.889719 26507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:36.889818 26507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "c96ca95d807c4cff9818e178eef311c0"
format_stamp: "Formatted at 2026-09-28 18:00:36 on dist-test-slave-z2t3"
server_key: "35b35c4b3ad163451eec7e527a508e37"
server_key_iv: "ce7b9d31a9d2e825286f0f54bb936ca5"
server_key_version: "encryptionkey@0"
I20260928 18:00:36.889901 26507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:36.901270 26507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:36.903626 26507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:36.903743 26507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:36.907681 26507 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45955
I20260928 18:00:36.907758 26574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45955 every 8 connection(s)
I20260928 18:00:36.908138 26507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:36.909286 26575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:36.912361 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26507
I20260928 18:00:36.912482 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:36.912679 5305 external_mini_cluster.cc:1428] Setting key 1f99766110fb496f34c65478507aa41d
I20260928 18:00:36.912801 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0: Bootstrap starting.
I20260928 18:00:36.913549 26575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:36.913877 26575 log.cc:824] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:36.914716 26575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0: No bootstrap required, opened a new log
I20260928 18:00:36.917328 26575 raft_consensus.cc:348] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } }
I20260928 18:00:36.917550 26575 raft_consensus.cc:374] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:36.917586 26575 raft_consensus.cc:729] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96ca95d807c4cff9818e178eef311c0, State: Initialized, Role: FOLLOWER
I20260928 18:00:36.917729 26575 consensus_queue.cc:260] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } }
I20260928 18:00:36.917811 26575 raft_consensus.cc:388] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:36.917870 26575 raft_consensus.cc:482] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:36.917920 26575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:36.918659 26575 raft_consensus.cc:504] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } }
I20260928 18:00:36.918800 26575 leader_election.cc:302] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c96ca95d807c4cff9818e178eef311c0; no voters:
I20260928 18:00:36.919037 26575 leader_election.cc:288] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:36.919083 26580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:36.919293 26580 raft_consensus.cc:686] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [term 1 LEADER]: Becoming Leader. State: Replica: c96ca95d807c4cff9818e178eef311c0, State: Running, Role: LEADER
I20260928 18:00:36.919371 26575 sys_catalog.cc:564] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:36.919467 26580 consensus_queue.cc:237] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } }
I20260928 18:00:36.919914 26582 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c96ca95d807c4cff9818e178eef311c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } } }
I20260928 18:00:36.920113 26582 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:36.920055 26580 sys_catalog.cc:455] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c96ca95d807c4cff9818e178eef311c0. Latest consensus state: current_term: 1 leader_uuid: "c96ca95d807c4cff9818e178eef311c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c96ca95d807c4cff9818e178eef311c0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45955 } } }
I20260928 18:00:36.920392 26580 sys_catalog.cc:458] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0 [sys.catalog]: This master's current role is: LEADER
Sep 28 18:00:36 dist-test-slave-z2t3 krb5kdc[26494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 18:00:36.920773 26589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:36.921294 26589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:36.922857 26589 catalog_manager.cc:1347] Generated new cluster ID: e32d178c89ee4570ab1ce3b141842ec0
I20260928 18:00:36.922912 26589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:36.925489 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:36.913309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:57007 (local address 127.0.0.1:45955)
0928 18:00:36.913530 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:36.913541 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:36.913578 (+ 37us) server_negotiation.cc:407] Connection header received
0928 18:00:36.914044 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:36.914051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:36.914103 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:36.914244 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:36.914978 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:36.916573 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:36.917400 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:36.918078 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:36.918212 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:36.921242 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:36.921245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:36.921246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:36.921256 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:36.923631 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:36.924106 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:36.924110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:36.924111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:36.924243 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:36.924495 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:36.924497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:36.924497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:36.924859 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:36.925042 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:36.925139 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 18:00:36.925269 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":120,"threads_started":1}
I20260928 18:00:36.926649 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:36.936957 26578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:36.928325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:53753 (local address 127.0.0.1:45955)
0928 18:00:36.928364 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:36.928366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:36.928442 (+ 76us) server_negotiation.cc:407] Connection header received
0928 18:00:36.928549 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:36.928550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:36.928561 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:36.928595 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:36.929424 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:36.930330 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:36.931504 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:36.931924 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:36.931975 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:36.933693 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:36.933694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:36.933694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:36.933703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:36.935391 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:36.935920 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:36.935922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:36.935922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:36.935985 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:36.936271 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:36.936272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:36.936273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:36.936415 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:36.936509 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:36.936779 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 18:00:36.936850 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 18:00:36.939640 26589 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:36.940243 26589 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:36.947297 26589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c96ca95d807c4cff9818e178eef311c0: Generated new TSK 0
I20260928 18:00:36.947531 26589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:36.951586 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26507
W20260928 18:00:36.956040 26492 connection.cc:535] client connection to 127.0.0.1:45955 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45955): Connection reset by peer (error 104)
2026-09-28T18:00:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:37 dist-test-slave-z2t3 krb5kdc[26606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:37 dist-test-slave-z2t3 krb5kdc[26606](info): set up 2 sockets
Sep 28 18:00:37 dist-test-slave-z2t3 krb5kdc[26606](info): commencing operation
krb5kdc: starting...
W20260928 18:00:37.980115 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:38 dist-test-slave-z2t3 krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:38.112083 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:38.194722 26619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:38.194979 26619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:38.195006 26619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:38.196244 26619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:38.196293 26619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:38.196308 26619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:38.196321 26619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:38.196333 26619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:38.197817 26619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:38.198025 26619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:38.198206 26619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:38.203238 26625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:38.203231 26627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:38.203231 26624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:38.203991 26619 server_base.cc:1034] running on GCE node
I20260928 18:00:38.204233 26619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:38.204577 26619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:38.205735 26619 hybrid_clock.cc:648] HybridClock initialized: now 1790618438205712 us; error 39 us; skew 500 ppm
Sep 28 18:00:38 dist-test-slave-z2t3 krb5kdc[26606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618438, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:38.208449 26619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:38.208979 26619 webserver.cc:460] Webserver started at http://127.0.0.1:33105/ using document root <none> and password file <none>
I20260928 18:00:38.209177 26619 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:38.209219 26619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:38.209321 26619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:38.210444 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "1352023271f349508179fb7abacf0ce0"
format_stamp: "Formatted at 2026-09-28 18:00:38 on dist-test-slave-z2t3"
server_key: "1f1f8129888c87eb1a3a675c16980311"
server_key_iv: "a5d71add6e2ba8229497b2ecd77293e6"
server_key_version: "encryptionkey@0"
I20260928 18:00:38.210763 26619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "1352023271f349508179fb7abacf0ce0"
format_stamp: "Formatted at 2026-09-28 18:00:38 on dist-test-slave-z2t3"
server_key: "1f1f8129888c87eb1a3a675c16980311"
server_key_iv: "a5d71add6e2ba8229497b2ecd77293e6"
server_key_version: "encryptionkey@0"
I20260928 18:00:38.212215 26619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:38.213050 26634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:38.213266 26619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:38.213390 26619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "1352023271f349508179fb7abacf0ce0"
format_stamp: "Formatted at 2026-09-28 18:00:38 on dist-test-slave-z2t3"
server_key: "1f1f8129888c87eb1a3a675c16980311"
server_key_iv: "a5d71add6e2ba8229497b2ecd77293e6"
server_key_version: "encryptionkey@0"
I20260928 18:00:38.213474 26619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:38.225099 26619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:38.227586 26619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:38.227768 26619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:38.231676 26619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46431
I20260928 18:00:38.231732 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46431 every 8 connection(s)
I20260928 18:00:38.232159 26619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:38.233124 26687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:38.235425 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0: Bootstrap starting.
I20260928 18:00:38.236099 26687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:38.236389 26687 log.cc:824] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:38.237174 26687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0: No bootstrap required, opened a new log
I20260928 18:00:38.237603 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26619
I20260928 18:00:38.237732 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:38.237934 5305 external_mini_cluster.cc:1428] Setting key 3535ab03a2a6adc130104d763cb2293b
I20260928 18:00:38.239838 26687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } }
I20260928 18:00:38.240060 26687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:38.240092 26687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1352023271f349508179fb7abacf0ce0, State: Initialized, Role: FOLLOWER
I20260928 18:00:38.240254 26687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } }
I20260928 18:00:38.240320 26687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:38.240342 26687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:38.240409 26687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:38.241082 26687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } }
I20260928 18:00:38.241207 26687 leader_election.cc:302] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1352023271f349508179fb7abacf0ce0; no voters:
I20260928 18:00:38.241410 26687 leader_election.cc:288] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:38.241434 26692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:38.241560 26692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [term 1 LEADER]: Becoming Leader. State: Replica: 1352023271f349508179fb7abacf0ce0, State: Running, Role: LEADER
I20260928 18:00:38.241727 26687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:38.241711 26692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } }
I20260928 18:00:38.242187 26694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1352023271f349508179fb7abacf0ce0. Latest consensus state: current_term: 1 leader_uuid: "1352023271f349508179fb7abacf0ce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } } }
I20260928 18:00:38.242283 26694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:38.242535 26693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1352023271f349508179fb7abacf0ce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1352023271f349508179fb7abacf0ce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46431 } } }
I20260928 18:00:38.242861 26693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:38.243070 26699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:38.243606 26699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:38.244994 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:38.238500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:50209 (local address 127.0.0.1:46431)
0928 18:00:38.238717 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:38.238725 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:38.238752 (+ 27us) server_negotiation.cc:407] Connection header received
0928 18:00:38.239252 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:38.239259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:38.239314 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:38.239435 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:38.239715 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:38.241132 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:38.242242 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:38.242911 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:38.242999 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:38.244251 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:38.244255 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:38.244256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:38.244467 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:38.244501 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:38.244513 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:38.244619 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:38.244707 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 18:00:38.244737 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260928 18:00:38.245401 26699 catalog_manager.cc:1347] Generated new cluster ID: 87be6465f2cc448da465318fd13a8d51
I20260928 18:00:38.245450 26699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:38.246100 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:38.251230 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:38.247539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:34687 (local address 127.0.0.1:46431)
0928 18:00:38.247593 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:38.247595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:38.247741 (+ 146us) server_negotiation.cc:407] Connection header received
0928 18:00:38.247790 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:38.247792 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:38.247803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:38.247919 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:38.248058 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:38.248824 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:38.249823 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:38.250153 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:38.250229 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:38.250967 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:38.250968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:38.250968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:38.250973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:38.250990 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:38.250994 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:38.251061 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:38.251134 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 18:00:38.251146 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 18:00:38.266130 26699 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:38.266757 26699 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:38.271657 26699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1352023271f349508179fb7abacf0ce0: Generated new TSK 0
I20260928 18:00:38.271945 26699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:38.282135 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26619
W20260928 18:00:38.286585 26604 connection.cc:535] client connection to 127.0.0.1:46431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46431): Connection reset by peer (error 104)
2026-09-28T18:00:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:38 dist-test-slave-z2t3 krb5kdc[26718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:38 dist-test-slave-z2t3 krb5kdc[26718](info): set up 2 sockets
Sep 28 18:00:38 dist-test-slave-z2t3 krb5kdc[26718](info): commencing operation
krb5kdc: starting...
W20260928 18:00:39.324112 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:39 dist-test-slave-z2t3 krb5kdc[26718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618439, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 18:00:39.450758 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:39.534317 26731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:39.534601 26731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:39.534644 26731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:39.535979 26731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:39.536046 26731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:39.536068 26731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:39.536089 26731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:39.536118 26731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:39.537690 26731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:39.538000 26731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:39.538264 26731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:39.543329 26736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:39.543387 26739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:39.543358 26737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:39.543718 26731 server_base.cc:1034] running on GCE node
I20260928 18:00:39.544040 26731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:39.544361 26731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:39.545562 26731 hybrid_clock.cc:648] HybridClock initialized: now 1790618439545531 us; error 59 us; skew 500 ppm
Sep 28 18:00:39 dist-test-slave-z2t3 krb5kdc[26718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:39.548447 26731 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 18:00:39.548979 26731 webserver.cc:460] Webserver started at http://127.0.0.1:44243/ using document root <none> and password file <none>
I20260928 18:00:39.549219 26731 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:39.549288 26731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:39.549440 26731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:39.550534 26731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b"
format_stamp: "Formatted at 2026-09-28 18:00:39 on dist-test-slave-z2t3"
server_key: "38ff3038cf86bc12b946c363c7f5f829"
server_key_iv: "4782ca8bdaec953914e0a436cf109631"
server_key_version: "encryptionkey@0"
I20260928 18:00:39.550915 26731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b"
format_stamp: "Formatted at 2026-09-28 18:00:39 on dist-test-slave-z2t3"
server_key: "38ff3038cf86bc12b946c363c7f5f829"
server_key_iv: "4782ca8bdaec953914e0a436cf109631"
server_key_version: "encryptionkey@0"
I20260928 18:00:39.552400 26731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:39.553301 26746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:39.553520 26731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:39.553653 26731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b"
format_stamp: "Formatted at 2026-09-28 18:00:39 on dist-test-slave-z2t3"
server_key: "38ff3038cf86bc12b946c363c7f5f829"
server_key_iv: "4782ca8bdaec953914e0a436cf109631"
server_key_version: "encryptionkey@0"
I20260928 18:00:39.553740 26731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:39.557421 26731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:39.559759 26731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:39.559985 26731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:39.564556 26731 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44539
I20260928 18:00:39.564584 26798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44539 every 8 connection(s)
I20260928 18:00:39.564973 26731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:39.565761 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26731
I20260928 18:00:39.565927 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:39.566147 5305 external_mini_cluster.cc:1428] Setting key 12d51a12e5ac9638936ce949eddfd203
I20260928 18:00:39.566354 26799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:39.568826 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:39.566762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:47289 (local address 127.0.0.1:44539)
0928 18:00:39.566974 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:39.566983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:39.567015 (+ 32us) server_negotiation.cc:407] Connection header received
0928 18:00:39.567498 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:39.567505 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:39.567536 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:39.567699 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:39.568395 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:39.568399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:39.568399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:39.568403 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:39.568450 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:39.568460 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:39.568539 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:39.568610 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 18:00:39.568662 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
I20260928 18:00:39.569872 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:39.569912 26799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b: Bootstrap starting.
I20260928 18:00:39.570523 26799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:39.570866 26799 log.cc:824] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:39.571588 26799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b: No bootstrap required, opened a new log
I20260928 18:00:39.572398 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:39.571244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.27:42173 (local address 127.0.0.1:44539)
0928 18:00:39.571298 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:39.571300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:39.571406 (+ 106us) server_negotiation.cc:407] Connection header received
0928 18:00:39.571476 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:39.571477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:39.571482 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:39.571526 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:39.572181 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:39.572183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:39.572183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:39.572187 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:39.572201 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:39.572205 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:39.572256 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:39.572320 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 18:00:39.572334 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 18:00:39.573693 26799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } }
I20260928 18:00:39.573834 26799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:39.573858 26799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bba6ec1db044dcf9dbf7bbb96800d8b, State: Initialized, Role: FOLLOWER
I20260928 18:00:39.574008 26799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } }
I20260928 18:00:39.574098 26799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:39.574152 26799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:39.574206 26799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:39.574777 26799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } }
I20260928 18:00:39.574867 26799 leader_election.cc:302] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bba6ec1db044dcf9dbf7bbb96800d8b; no voters:
I20260928 18:00:39.575040 26799 leader_election.cc:288] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:39.575101 26809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:39.575285 26809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [term 1 LEADER]: Becoming Leader. State: Replica: 0bba6ec1db044dcf9dbf7bbb96800d8b, State: Running, Role: LEADER
I20260928 18:00:39.575320 26799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:39.575403 26809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } }
I20260928 18:00:39.575773 26810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } } }
I20260928 18:00:39.575796 26811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bba6ec1db044dcf9dbf7bbb96800d8b. Latest consensus state: current_term: 1 leader_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bba6ec1db044dcf9dbf7bbb96800d8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44539 } } }
I20260928 18:00:39.575963 26810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:39.576009 26811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:39.576222 26818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:39.577109 26818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:39.578871 26818 catalog_manager.cc:1347] Generated new cluster ID: a2679d6f88ab4618bbaf034eb655a2ea
I20260928 18:00:39.578930 26818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:39.630932 26818 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:39.631181 26818 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:39.637488 26818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bba6ec1db044dcf9dbf7bbb96800d8b: Generated new TSK 0
I20260928 18:00:39.637717 26818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:39.644739 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26731
2026-09-28T18:00:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:39 dist-test-slave-z2t3 krb5kdc[26830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:39 dist-test-slave-z2t3 krb5kdc[26830](info): set up 2 sockets
Sep 28 18:00:39 dist-test-slave-z2t3 krb5kdc[26830](info): commencing operation
krb5kdc: starting...
W20260928 18:00:40.692164 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:40 dist-test-slave-z2t3 krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:40.818849 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:45235
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:39785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:45235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:40.900187 26843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:40.900449 26843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:40.900490 26843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:40.901695 26843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:40.901772 26843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:40.901794 26843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:40.901816 26843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:40.901841 26843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:40.903360 26843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:39785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:45235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:45235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:40.903642 26843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:40.903919 26843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:40.908857 26851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:40.908928 26848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:40.909050 26843 server_base.cc:1034] running on GCE node
W20260928 18:00:40.908850 26849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:40.909390 26843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:40.909708 26843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:40.910893 26843 hybrid_clock.cc:648] HybridClock initialized: now 1790618440910867 us; error 35 us; skew 500 ppm
Sep 28 18:00:40 dist-test-slave-z2t3 krb5kdc[26830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618440, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:40.913725 26843 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:40.914206 26843 webserver.cc:460] Webserver started at http://127.5.46.126:38619/ using document root <none> and password file <none>
I20260928 18:00:40.914445 26843 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:40.914515 26843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:40.914669 26843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:40.915835 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "4580fe0c25b2491a8765d5dd3f7f8897"
format_stamp: "Formatted at 2026-09-28 18:00:40 on dist-test-slave-z2t3"
server_key: "252893336615312e9885b9d9cc3a5135"
server_key_iv: "62288f05bd88b5f20b11b27e47691bd4"
server_key_version: "encryptionkey@0"
I20260928 18:00:40.916290 26843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "4580fe0c25b2491a8765d5dd3f7f8897"
format_stamp: "Formatted at 2026-09-28 18:00:40 on dist-test-slave-z2t3"
server_key: "252893336615312e9885b9d9cc3a5135"
server_key_iv: "62288f05bd88b5f20b11b27e47691bd4"
server_key_version: "encryptionkey@0"
I20260928 18:00:40.917665 26843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 18:00:40.918371 26858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:40.918547 26843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:40.918648 26843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "4580fe0c25b2491a8765d5dd3f7f8897"
format_stamp: "Formatted at 2026-09-28 18:00:40 on dist-test-slave-z2t3"
server_key: "252893336615312e9885b9d9cc3a5135"
server_key_iv: "62288f05bd88b5f20b11b27e47691bd4"
server_key_version: "encryptionkey@0"
I20260928 18:00:40.918736 26843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:40.963467 26843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:40.965729 26843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:40.965909 26843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:40.969949 26843 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:45235
I20260928 18:00:40.970024 26910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:45235 every 8 connection(s)
I20260928 18:00:40.970405 26843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:40.971330 26911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:40.973697 26911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: Bootstrap starting.
I20260928 18:00:40.974231 26911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:40.974414 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26843
I20260928 18:00:40.974548 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:40.974568 26911 log.cc:824] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:40.974808 5305 external_mini_cluster.cc:1428] Setting key 0f02b9194c3f1b04b2af93f3e6107b1f
I20260928 18:00:40.975487 26911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: No bootstrap required, opened a new log
I20260928 18:00:40.977281 26911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } }
I20260928 18:00:40.977427 26911 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:40.977448 26911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4580fe0c25b2491a8765d5dd3f7f8897, State: Initialized, Role: FOLLOWER
I20260928 18:00:40.977581 26911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } }
I20260928 18:00:40.977689 26911 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:40.977735 26911 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:40.977807 26911 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:40.978760 26911 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } }
I20260928 18:00:40.978883 26911 leader_election.cc:302] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4580fe0c25b2491a8765d5dd3f7f8897; no voters:
I20260928 18:00:40.979099 26911 leader_election.cc:288] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:40.979125 26916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:40.979234 26916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [term 1 LEADER]: Becoming Leader. State: Replica: 4580fe0c25b2491a8765d5dd3f7f8897, State: Running, Role: LEADER
I20260928 18:00:40.979380 26911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:40.979383 26916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } }
I20260928 18:00:40.979928 26917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } } }
I20260928 18:00:40.980020 26917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:40.980180 26918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4580fe0c25b2491a8765d5dd3f7f8897. Latest consensus state: current_term: 1 leader_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4580fe0c25b2491a8765d5dd3f7f8897" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 45235 } } }
I20260928 18:00:40.980237 26918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897 [sys.catalog]: This master's current role is: LEADER
W20260928 18:00:40.981369 26931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:40.981469 26931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:40.981539 26924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:40.981702 26924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:40.983346 26924 catalog_manager.cc:1347] Generated new cluster ID: 4f15901b1b58498f9bc1d4bd97191c80
I20260928 18:00:40.983399 26924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 18:00:40 dist-test-slave-z2t3 krb5kdc[26830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:40.988006 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:40.975409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56931 (local address 127.5.46.126:45235)
0928 18:00:40.975637 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:40.975653 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:40.975679 (+ 26us) server_negotiation.cc:407] Connection header received
0928 18:00:40.977289 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:40.977298 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:40.977357 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:40.977591 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:40.978379 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:40.979519 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:40.980624 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:40.981215 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:40.981352 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:40.984208 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:40.984213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:40.984213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:40.984229 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:40.986282 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:40.986700 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:40.986704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:40.986705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:40.986846 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:40.987089 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:40.987090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:40.987091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:40.987350 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:40.987535 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:40.987622 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 18:00:40.987745 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260928 18:00:40.988937 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:40.996222 26924 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:40.996913 26924 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:40.997594 26915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:40.990374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35439 (local address 127.5.46.126:45235)
0928 18:00:40.990409 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:40.990411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:40.990531 (+ 120us) server_negotiation.cc:407] Connection header received
0928 18:00:40.990603 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:40.990604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:40.990614 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:40.990665 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:40.991187 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:40.991702 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:40.992468 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:40.992701 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:40.992782 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:40.994269 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:40.994271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:40.994271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:40.994280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:40.996246 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:40.996682 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:40.996683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:40.996683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:40.996749 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:40.996993 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:40.996994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:40.996995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:40.997119 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:40.997242 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:40.997399 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 18:00:40.997470 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 18:00:41.008672 26924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4580fe0c25b2491a8765d5dd3f7f8897: Generated new TSK 0
I20260928 18:00:41.008903 26924 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:41.013795 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26843
W20260928 18:00:41.018298 26828 connection.cc:535] client connection to 127.5.46.126:45235 recv error: Network error: failed to read from TLS socket (remote: 127.5.46.126:45235): Connection reset by peer (error 104)
2026-09-28T18:00:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:41 dist-test-slave-z2t3 krb5kdc[26942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:41 dist-test-slave-z2t3 krb5kdc[26942](info): set up 2 sockets
Sep 28 18:00:41 dist-test-slave-z2t3 krb5kdc[26942](info): commencing operation
krb5kdc: starting...
W20260928 18:00:42.052055 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:42 dist-test-slave-z2t3 krb5kdc[26942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:42.183135 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33439
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:33439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:42.266110 26955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:42.266381 26955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:42.266424 26955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:42.268018 26955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:42.268090 26955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:42.268121 26955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:42.268147 26955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:42.268177 26955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:42.269891 26955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:33439
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:33439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:42.270115 26955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:42.270299 26955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:42.275240 26961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:42.275264 26960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:42.275240 26963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:42.275704 26955 server_base.cc:1034] running on GCE node
I20260928 18:00:42.276034 26955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:42.276350 26955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:42.277530 26955 hybrid_clock.cc:648] HybridClock initialized: now 1790618442277515 us; error 51 us; skew 500 ppm
Sep 28 18:00:42 dist-test-slave-z2t3 krb5kdc[26942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618442, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:42.285738 26955 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:42.286206 26955 webserver.cc:460] Webserver started at http://127.5.46.126:34707/ using document root <none> and password file <none>
I20260928 18:00:42.286494 26955 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:42.286546 26955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:42.286686 26955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:42.287822 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "2e55c375eee143eba6310334843aaa10"
format_stamp: "Formatted at 2026-09-28 18:00:42 on dist-test-slave-z2t3"
server_key: "d82fa497b5bb171f11850c278a524ef0"
server_key_iv: "5f89dbbd12fcd0d6ba082137814c79ba"
server_key_version: "encryptionkey@0"
I20260928 18:00:42.288282 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "2e55c375eee143eba6310334843aaa10"
format_stamp: "Formatted at 2026-09-28 18:00:42 on dist-test-slave-z2t3"
server_key: "d82fa497b5bb171f11850c278a524ef0"
server_key_iv: "5f89dbbd12fcd0d6ba082137814c79ba"
server_key_version: "encryptionkey@0"
I20260928 18:00:42.289707 26955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:42.290553 26970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:42.290829 26955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:42.290944 26955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "2e55c375eee143eba6310334843aaa10"
format_stamp: "Formatted at 2026-09-28 18:00:42 on dist-test-slave-z2t3"
server_key: "d82fa497b5bb171f11850c278a524ef0"
server_key_iv: "5f89dbbd12fcd0d6ba082137814c79ba"
server_key_version: "encryptionkey@0"
I20260928 18:00:42.291025 26955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:42.320961 26955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:42.323542 26955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:42.323717 26955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:42.327509 26955 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:33439
I20260928 18:00:42.327575 27022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:33439 every 8 connection(s)
I20260928 18:00:42.327934 26955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:42.328558 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 26955
I20260928 18:00:42.328683 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:42.328871 5305 external_mini_cluster.cc:1428] Setting key f2058ebd9f913d353baf260da07864da
I20260928 18:00:42.328919 27023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:42.332245 27023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: Bootstrap starting.
I20260928 18:00:42.332880 27023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:42.333237 27023 log.cc:824] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:42.333942 27023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: No bootstrap required, opened a new log
I20260928 18:00:42.334712 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:42.329389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59397 (local address 127.5.46.126:33439)
0928 18:00:42.329586 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:42.329594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:42.329620 (+ 26us) server_negotiation.cc:407] Connection header received
0928 18:00:42.330018 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:42.330025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:42.330079 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:42.330229 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:42.330474 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:42.331359 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:42.332539 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:42.333029 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:42.333118 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:42.333967 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:42.333972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:42.333972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:42.334206 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:42.334240 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:42.334251 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:42.334356 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:42.334479 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 18:00:42.334510 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":64,"threads_started":1}
I20260928 18:00:42.336247 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:42.336432 27023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } }
I20260928 18:00:42.336589 27023 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:42.336635 27023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e55c375eee143eba6310334843aaa10, State: Initialized, Role: FOLLOWER
I20260928 18:00:42.336757 27023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } }
I20260928 18:00:42.336853 27023 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:42.336925 27023 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:42.336990 27023 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:42.337631 27023 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } }
I20260928 18:00:42.337750 27023 leader_election.cc:302] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e55c375eee143eba6310334843aaa10; no voters:
I20260928 18:00:42.337988 27023 leader_election.cc:288] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:42.338066 27033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:42.338222 27033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [term 1 LEADER]: Becoming Leader. State: Replica: 2e55c375eee143eba6310334843aaa10, State: Running, Role: LEADER
I20260928 18:00:42.338310 27023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:42.338436 27033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } }
I20260928 18:00:42.338822 27033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e55c375eee143eba6310334843aaa10. Latest consensus state: current_term: 1 leader_uuid: "2e55c375eee143eba6310334843aaa10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } } }
I20260928 18:00:42.338857 27035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e55c375eee143eba6310334843aaa10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e55c375eee143eba6310334843aaa10" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 33439 } } }
I20260928 18:00:42.339028 27035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:42.338941 27033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10 [sys.catalog]: This master's current role is: LEADER
W20260928 18:00:42.340314 27048 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:42.340372 27048 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:42.340425 27042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:42.340616 27042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:42.342000 27026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:42.337849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38627 (local address 127.5.46.126:33439)
0928 18:00:42.337895 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:42.337897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:42.338030 (+ 133us) server_negotiation.cc:407] Connection header received
0928 18:00:42.338082 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:42.338083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:42.338095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:42.338156 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:42.338410 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:42.339110 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:42.340313 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:42.340607 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:42.340769 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:42.341762 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:42.341764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:42.341764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:42.341769 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:42.341785 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:42.341790 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:42.341847 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:42.341907 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 18:00:42.341916 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:42.342512 27042 catalog_manager.cc:1347] Generated new cluster ID: 13825285756843b8a93bdf921a0f55b8
I20260928 18:00:42.342582 27042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:42.350451 27042 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:42.351024 27042 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:42.361970 27042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e55c375eee143eba6310334843aaa10: Generated new TSK 0
I20260928 18:00:42.362201 27042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:42.366107 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 26955
W20260928 18:00:42.370391 26940 connection.cc:535] client connection to 127.5.46.126:33439 recv error: Network error: failed to read from TLS socket (remote: 127.5.46.126:33439): Connection reset by peer (error 104)
2026-09-28T18:00:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:42 dist-test-slave-z2t3 krb5kdc[27054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:42 dist-test-slave-z2t3 krb5kdc[27054](info): set up 2 sockets
Sep 28 18:00:42 dist-test-slave-z2t3 krb5kdc[27054](info): commencing operation
krb5kdc: starting...
W20260928 18:00:43.428079 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:43 dist-test-slave-z2t3 krb5kdc[27054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:43.551251 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37387
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:38565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:37387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:43.633088 27067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:43.633313 27067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:43.633338 27067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:43.634498 27067 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:43.634542 27067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:43.634557 27067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:43.634569 27067 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:43.634581 27067 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:43.636065 27067 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:38565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:37387
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:37387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:43.636267 27067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:43.636442 27067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 18:00:43.641409 27067 server_base.cc:1034] running on GCE node
W20260928 18:00:43.641398 27072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:43.641389 27073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:43.641382 27075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:43.641916 27067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:43.642216 27067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:43.643391 27067 hybrid_clock.cc:648] HybridClock initialized: now 1790618443643398 us; error 46 us; skew 500 ppm
Sep 28 18:00:43 dist-test-slave-z2t3 krb5kdc[27054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618443, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:43.646081 27067 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:43.646569 27067 webserver.cc:460] Webserver started at http://127.5.46.126:41073/ using document root <none> and password file <none>
I20260928 18:00:43.646806 27067 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:43.646881 27067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:43.647027 27067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:43.648159 27067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "0da48bf5fad74a4dad2540903175c517"
format_stamp: "Formatted at 2026-09-28 18:00:43 on dist-test-slave-z2t3"
server_key: "f8f2407ace170986f02b1096d08bd51a"
server_key_iv: "3e0f09dd6a8ccfac9d06a152107bc3ab"
server_key_version: "encryptionkey@0"
I20260928 18:00:43.648519 27067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "0da48bf5fad74a4dad2540903175c517"
format_stamp: "Formatted at 2026-09-28 18:00:43 on dist-test-slave-z2t3"
server_key: "f8f2407ace170986f02b1096d08bd51a"
server_key_iv: "3e0f09dd6a8ccfac9d06a152107bc3ab"
server_key_version: "encryptionkey@0"
I20260928 18:00:43.649889 27067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:43.650646 27082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:43.650816 27067 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:43.650931 27067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "0da48bf5fad74a4dad2540903175c517"
format_stamp: "Formatted at 2026-09-28 18:00:43 on dist-test-slave-z2t3"
server_key: "f8f2407ace170986f02b1096d08bd51a"
server_key_iv: "3e0f09dd6a8ccfac9d06a152107bc3ab"
server_key_version: "encryptionkey@0"
I20260928 18:00:43.651008 27067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:43.654515 27067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:43.656967 27067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:43.657133 27067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:43.661265 27067 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:37387
I20260928 18:00:43.661295 27134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:37387 every 8 connection(s)
I20260928 18:00:43.661633 27067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:43.662578 27135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:43.664870 27135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517: Bootstrap starting.
I20260928 18:00:43.665445 27135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:43.665732 27135 log.cc:824] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:43.666216 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27067
I20260928 18:00:43.666342 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:43.666517 5305 external_mini_cluster.cc:1428] Setting key d2d86a50e43d23acda013abcfaa1ff30
I20260928 18:00:43.666543 27135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517: No bootstrap required, opened a new log
I20260928 18:00:43.669003 27135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } }
I20260928 18:00:43.669214 27135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:43.669262 27135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da48bf5fad74a4dad2540903175c517, State: Initialized, Role: FOLLOWER
I20260928 18:00:43.669313 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:43.667052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60389 (local address 127.5.46.126:37387)
0928 18:00:43.667218 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:43.667227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:43.667263 (+ 36us) server_negotiation.cc:407] Connection header received
0928 18:00:43.667912 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:43.667921 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:43.667956 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:43.668080 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:43.668680 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:43.668684 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:43.668684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:43.668687 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:43.668722 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:43.668733 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:43.668785 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:43.668871 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 18:00:43.668931 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
I20260928 18:00:43.669406 27135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } }
I20260928 18:00:43.669497 27135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:43.669528 27135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:43.669574 27135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:43.670215 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:43.670257 27135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } }
I20260928 18:00:43.670354 27135 leader_election.cc:302] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0da48bf5fad74a4dad2540903175c517; no voters:
I20260928 18:00:43.670501 27135 leader_election.cc:288] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:43.670553 27140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:43.670727 27140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [term 1 LEADER]: Becoming Leader. State: Replica: 0da48bf5fad74a4dad2540903175c517, State: Running, Role: LEADER
I20260928 18:00:43.670735 27135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:43.670885 27140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } }
I20260928 18:00:43.671219 27140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0da48bf5fad74a4dad2540903175c517. Latest consensus state: current_term: 1 leader_uuid: "0da48bf5fad74a4dad2540903175c517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } } }
I20260928 18:00:43.671226 27141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0da48bf5fad74a4dad2540903175c517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da48bf5fad74a4dad2540903175c517" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 37387 } } }
I20260928 18:00:43.671319 27140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:43.671322 27141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:43.672070 27147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:43.672649 27147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:43.673332 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:43.671866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36749 (local address 127.5.46.126:37387)
0928 18:00:43.672003 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:43.672005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:43.672277 (+ 272us) server_negotiation.cc:407] Connection header received
0928 18:00:43.672389 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:43.672390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:43.672396 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:43.672458 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:43.673154 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:43.673155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:43.673155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:43.673158 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:43.673172 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:43.673175 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:43.673223 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:43.673277 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 18:00:43.673288 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260928 18:00:43.674513 27147 catalog_manager.cc:1347] Generated new cluster ID: cc2811cce50a4062886fdfa871db5553
I20260928 18:00:43.674571 27147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:43.692533 27147 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:43.692750 27147 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:43.697088 27147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0da48bf5fad74a4dad2540903175c517: Generated new TSK 0
I20260928 18:00:43.697306 27147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:43.702991 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27067
2026-09-28T18:00:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:43 dist-test-slave-z2t3 krb5kdc[27166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:43 dist-test-slave-z2t3 krb5kdc[27166](info): set up 2 sockets
Sep 28 18:00:43 dist-test-slave-z2t3 krb5kdc[27166](info): commencing operation
krb5kdc: starting...
W20260928 18:00:44.740091 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:44 dist-test-slave-z2t3 krb5kdc[27166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:44.860920 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46417
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:33127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:44.943629 27179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:44.943897 27179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:44.943939 27179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:44.945163 27179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:44.945209 27179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:44.945225 27179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:44.945237 27179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:44.945250 27179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:44.946851 27179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:33127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:44.947129 27179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:44.947388 27179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:44.952416 27187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:44.952479 27185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:44.952416 27184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:44.952602 27179 server_base.cc:1034] running on GCE node
I20260928 18:00:44.952823 27179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:44.953087 27179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:44.954237 27179 hybrid_clock.cc:648] HybridClock initialized: now 1790618444954215 us; error 37 us; skew 500 ppm
Sep 28 18:00:44 dist-test-slave-z2t3 krb5kdc[27166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618444, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:44.957118 27179 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:44.957600 27179 webserver.cc:460] Webserver started at http://127.5.46.126:43095/ using document root <none> and password file <none>
I20260928 18:00:44.957830 27179 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:44.957880 27179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:44.958045 27179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:44.959095 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "eac60e61de3940bc9dfaca43e7da2115"
format_stamp: "Formatted at 2026-09-28 18:00:44 on dist-test-slave-z2t3"
server_key: "6bb2625dcf0f3eff5645fd3f711b01d9"
server_key_iv: "614aea6cc620792c67d7556a31ebddd7"
server_key_version: "encryptionkey@0"
I20260928 18:00:44.959451 27179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "eac60e61de3940bc9dfaca43e7da2115"
format_stamp: "Formatted at 2026-09-28 18:00:44 on dist-test-slave-z2t3"
server_key: "6bb2625dcf0f3eff5645fd3f711b01d9"
server_key_iv: "614aea6cc620792c67d7556a31ebddd7"
server_key_version: "encryptionkey@0"
I20260928 18:00:44.960865 27179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:44.961632 27194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:44.961848 27179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:44.961956 27179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "eac60e61de3940bc9dfaca43e7da2115"
format_stamp: "Formatted at 2026-09-28 18:00:44 on dist-test-slave-z2t3"
server_key: "6bb2625dcf0f3eff5645fd3f711b01d9"
server_key_iv: "614aea6cc620792c67d7556a31ebddd7"
server_key_version: "encryptionkey@0"
I20260928 18:00:44.962028 27179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:44.970575 27179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:44.972976 27179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:44.973145 27179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:44.977474 27179 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46417
I20260928 18:00:44.977550 27246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46417 every 8 connection(s)
I20260928 18:00:44.977921 27179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:44.978788 27247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:44.981168 27247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115: Bootstrap starting.
I20260928 18:00:44.981751 27247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:44.982030 27247 log.cc:824] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:44.982736 27247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115: No bootstrap required, opened a new log
I20260928 18:00:44.984418 27247 raft_consensus.cc:348] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } }
I20260928 18:00:44.984555 27247 raft_consensus.cc:374] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:44.984576 27247 raft_consensus.cc:729] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac60e61de3940bc9dfaca43e7da2115, State: Initialized, Role: FOLLOWER
I20260928 18:00:44.984743 27247 consensus_queue.cc:260] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } }
I20260928 18:00:44.984804 27247 raft_consensus.cc:388] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:44.984828 27247 raft_consensus.cc:482] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:44.984901 27247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:44.985772 27247 raft_consensus.cc:504] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } }
I20260928 18:00:44.985859 27247 leader_election.cc:302] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eac60e61de3940bc9dfaca43e7da2115; no voters:
I20260928 18:00:44.986021 27247 leader_election.cc:288] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:44.986050 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27179
I20260928 18:00:44.986075 27250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:44.986164 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:44.986222 27250 raft_consensus.cc:686] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [term 1 LEADER]: Becoming Leader. State: Replica: eac60e61de3940bc9dfaca43e7da2115, State: Running, Role: LEADER
I20260928 18:00:44.986303 27247 sys_catalog.cc:564] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:44.986362 5305 external_mini_cluster.cc:1428] Setting key 41984877e52514d57c6fd7155b312bf3
I20260928 18:00:44.986320 27250 consensus_queue.cc:237] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } }
I20260928 18:00:44.986756 27250 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eac60e61de3940bc9dfaca43e7da2115. Latest consensus state: current_term: 1 leader_uuid: "eac60e61de3940bc9dfaca43e7da2115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } } }
I20260928 18:00:44.986860 27250 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:44.986769 27251 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eac60e61de3940bc9dfaca43e7da2115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac60e61de3940bc9dfaca43e7da2115" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46417 } } }
I20260928 18:00:44.987052 27251 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:44.987112 27259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:44.987623 27259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:44.992470 27259 catalog_manager.cc:1347] Generated new cluster ID: ad005ab738df42399709d615a68b3c80
I20260928 18:00:44.992532 27259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 18:00:44 dist-test-slave-z2t3 krb5kdc[27166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:44.997563 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:44.987007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50891 (local address 127.5.46.126:46417)
0928 18:00:44.988076 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:44.988084 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:44.988117 (+ 33us) server_negotiation.cc:407] Connection header received
0928 18:00:44.988308 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:44.988318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:44.988382 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:44.988540 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:44.989201 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:44.990112 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:44.990876 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:44.991324 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:44.991451 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:44.994008 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:44.994018 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:44.994018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:44.994031 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:44.995908 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:44.996302 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:44.996305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:44.996306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:44.996422 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:44.996648 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:44.996649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:44.996649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:44.996911 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:44.997138 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:44.997208 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 18:00:44.997378 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1019,"thread_start_us":81,"threads_started":1}
I20260928 18:00:44.998526 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:45.010351 27259 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:45.010974 27261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:45.000106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41197 (local address 127.5.46.126:46417)
0928 18:00:45.000140 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:45.000141 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:45.004150 (+ 4009us) server_negotiation.cc:407] Connection header received
0928 18:00:45.004263 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:45.004264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:45.004278 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:45.004331 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:45.004972 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:45.005503 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:45.006398 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:45.006630 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:45.006722 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:45.008098 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:45.008100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:45.008100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:45.008108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:45.009669 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:45.010072 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:45.010074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:45.010074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:45.010135 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:45.010381 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:45.010382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:45.010382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:45.010514 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:45.010601 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:45.010763 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 18:00:45.010852 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 18:00:45.011117 27259 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:45.022276 27259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eac60e61de3940bc9dfaca43e7da2115: Generated new TSK 0
I20260928 18:00:45.022542 27259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:45.027312 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27179
2026-09-28T18:00:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:45 dist-test-slave-z2t3 krb5kdc[27278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:45 dist-test-slave-z2t3 krb5kdc[27278](info): set up 2 sockets
Sep 28 18:00:45 dist-test-slave-z2t3 krb5kdc[27278](info): commencing operation
krb5kdc: starting...
W20260928 18:00:46.072005 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:46 dist-test-slave-z2t3 krb5kdc[27278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:46.199469 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42885
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:35381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:46.282567 27291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:46.282840 27291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:46.282888 27291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:46.284179 27291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:46.284247 27291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:46.284271 27291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:46.284302 27291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:46.284332 27291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:46.285903 27291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:35381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:46.286204 27291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:46.286449 27291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:46.291668 27297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:46.291605 27299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:46.291702 27291 server_base.cc:1034] running on GCE node
W20260928 18:00:46.291605 27296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:46.292098 27291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:46.292454 27291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:46.293622 27291 hybrid_clock.cc:648] HybridClock initialized: now 1790618446293618 us; error 37 us; skew 500 ppm
Sep 28 18:00:46 dist-test-slave-z2t3 krb5kdc[27278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618446, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:46.296162 27291 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:46.296631 27291 webserver.cc:460] Webserver started at http://127.5.46.126:40157/ using document root <none> and password file <none>
I20260928 18:00:46.296860 27291 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:46.296931 27291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:46.297087 27291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:46.298228 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "57bfdf1917064495a1a58a8b7fc90f65"
format_stamp: "Formatted at 2026-09-28 18:00:46 on dist-test-slave-z2t3"
server_key: "303db08453302d7014040a5f48c5b912"
server_key_iv: "21c9e0fc7d70c67daeb748d4a7856e33"
server_key_version: "encryptionkey@0"
I20260928 18:00:46.298604 27291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "57bfdf1917064495a1a58a8b7fc90f65"
format_stamp: "Formatted at 2026-09-28 18:00:46 on dist-test-slave-z2t3"
server_key: "303db08453302d7014040a5f48c5b912"
server_key_iv: "21c9e0fc7d70c67daeb748d4a7856e33"
server_key_version: "encryptionkey@0"
I20260928 18:00:46.299983 27291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:46.300740 27306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:46.300906 27291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:46.301005 27291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "57bfdf1917064495a1a58a8b7fc90f65"
format_stamp: "Formatted at 2026-09-28 18:00:46 on dist-test-slave-z2t3"
server_key: "303db08453302d7014040a5f48c5b912"
server_key_iv: "21c9e0fc7d70c67daeb748d4a7856e33"
server_key_version: "encryptionkey@0"
I20260928 18:00:46.301095 27291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:46.318432 27291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:46.321012 27291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:46.321205 27291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:46.325639 27291 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:42885
I20260928 18:00:46.325676 27358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:42885 every 8 connection(s)
I20260928 18:00:46.326057 27291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:46.327010 27359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:46.329432 27359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65: Bootstrap starting.
I20260928 18:00:46.329983 27359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:46.330219 27359 log.cc:824] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:46.330892 27359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65: No bootstrap required, opened a new log
I20260928 18:00:46.332589 27359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } }
I20260928 18:00:46.332737 27359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:46.332762 27359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bfdf1917064495a1a58a8b7fc90f65, State: Initialized, Role: FOLLOWER
I20260928 18:00:46.332916 27359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } }
I20260928 18:00:46.333002 27359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:46.333027 27359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:46.333057 27359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:46.333892 27359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } }
I20260928 18:00:46.333981 27359 leader_election.cc:302] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57bfdf1917064495a1a58a8b7fc90f65; no voters:
I20260928 18:00:46.334120 27359 leader_election.cc:288] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:46.334211 27362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:46.334396 27362 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [term 1 LEADER]: Becoming Leader. State: Replica: 57bfdf1917064495a1a58a8b7fc90f65, State: Running, Role: LEADER
I20260928 18:00:46.334432 27359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:46.334493 27362 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } }
I20260928 18:00:46.334921 27363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57bfdf1917064495a1a58a8b7fc90f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } } }
I20260928 18:00:46.335003 27363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:46.335034 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27291
I20260928 18:00:46.335146 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:46.335314 5305 external_mini_cluster.cc:1428] Setting key 1a179aae791a075a3e2e207562ef9338
I20260928 18:00:46.335646 27371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:46.336126 27364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57bfdf1917064495a1a58a8b7fc90f65. Latest consensus state: current_term: 1 leader_uuid: "57bfdf1917064495a1a58a8b7fc90f65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57bfdf1917064495a1a58a8b7fc90f65" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42885 } } }
I20260928 18:00:46.336226 27364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:46.336300 27371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:46.338097 27371 catalog_manager.cc:1347] Generated new cluster ID: 3e890d4f6907475b863974a71ecfb3c8
I20260928 18:00:46.338158 27371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:46.341331 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:46.336010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42763 (local address 127.5.46.126:42885)
0928 18:00:46.336343 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:46.336354 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:46.336390 (+ 36us) server_negotiation.cc:407] Connection header received
0928 18:00:46.336590 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:46.336598 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:46.336658 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:46.336801 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:46.337067 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:46.338397 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:46.339270 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:46.339731 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:46.339904 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:46.340656 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:46.340660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:46.340661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:46.340851 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:46.340884 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:46.340897 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:46.340979 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:46.341055 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 18:00:46.341128 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":111,"threads_started":1}
I20260928 18:00:46.342314 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:46.346839 27381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:46.343768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52349 (local address 127.5.46.126:42885)
0928 18:00:46.343833 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:46.343836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:46.343933 (+ 97us) server_negotiation.cc:407] Connection header received
0928 18:00:46.344006 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:46.344007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:46.344019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:46.344075 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:46.344229 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:46.344776 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:46.345563 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:46.345778 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:46.345858 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:46.346579 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:46.346580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:46.346581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:46.346586 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:46.346601 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:46.346606 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:46.346653 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:46.346709 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 18:00:46.346736 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 18:00:46.356997 27371 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:46.357616 27371 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:46.365217 27371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57bfdf1917064495a1a58a8b7fc90f65: Generated new TSK 0
I20260928 18:00:46.365471 27371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:46.373843 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27291
2026-09-28T18:00:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1346 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:46 dist-test-slave-z2t3 krb5kdc[27390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:46 dist-test-slave-z2t3 krb5kdc[27390](info): set up 2 sockets
Sep 28 18:00:46 dist-test-slave-z2t3 krb5kdc[27390](info): commencing operation
krb5kdc: starting...
W20260928 18:00:47.436137 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:47 dist-test-slave-z2t3 krb5kdc[27390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:47.564134 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44471
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:33199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:44471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:47.645789 27403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:47.646010 27403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:47.646035 27403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:47.647255 27403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:47.647303 27403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:47.647317 27403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:47.647330 27403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:47.647392 27403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:47.649076 27403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:33199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:44471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:44471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:47.649341 27403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:47.649590 27403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 18:00:47.654594 27403 server_base.cc:1034] running on GCE node
W20260928 18:00:47.654527 27411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:47.654527 27408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:47.654551 27409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:47.655028 27403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:47.655316 27403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:47.656461 27403 hybrid_clock.cc:648] HybridClock initialized: now 1790618447656439 us; error 39 us; skew 500 ppm
Sep 28 18:00:47 dist-test-slave-z2t3 krb5kdc[27390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618447, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:47.664012 27403 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:47.664526 27403 webserver.cc:460] Webserver started at http://127.5.46.126:34639/ using document root <none> and password file <none>
I20260928 18:00:47.664721 27403 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:47.664762 27403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:47.664856 27403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:47.665944 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "3164b18434854863b960bd42336900b1"
format_stamp: "Formatted at 2026-09-28 18:00:47 on dist-test-slave-z2t3"
server_key: "81685595f4bec266ba771fe2cccf1963"
server_key_iv: "35ea360b08baf899f581c4e88ed7d3f4"
server_key_version: "encryptionkey@0"
I20260928 18:00:47.666254 27403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "3164b18434854863b960bd42336900b1"
format_stamp: "Formatted at 2026-09-28 18:00:47 on dist-test-slave-z2t3"
server_key: "81685595f4bec266ba771fe2cccf1963"
server_key_iv: "35ea360b08baf899f581c4e88ed7d3f4"
server_key_version: "encryptionkey@0"
I20260928 18:00:47.667627 27403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:47.668597 27418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:47.668828 27403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:47.668927 27403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "3164b18434854863b960bd42336900b1"
format_stamp: "Formatted at 2026-09-28 18:00:47 on dist-test-slave-z2t3"
server_key: "81685595f4bec266ba771fe2cccf1963"
server_key_iv: "35ea360b08baf899f581c4e88ed7d3f4"
server_key_version: "encryptionkey@0"
I20260928 18:00:47.668990 27403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:47.672428 27403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:47.674739 27403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:47.674858 27403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:47.678704 27403 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:44471
I20260928 18:00:47.678735 27470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:44471 every 8 connection(s)
I20260928 18:00:47.679080 27403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:47.680160 27471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:47.682459 27471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1: Bootstrap starting.
I20260928 18:00:47.683054 27471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:47.683346 27471 log.cc:824] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:47.684185 27471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1: No bootstrap required, opened a new log
I20260928 18:00:47.685755 27471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } }
I20260928 18:00:47.685925 27471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 18:00:47.685978 27471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3164b18434854863b960bd42336900b1, State: Initialized, Role: FOLLOWER
I20260928 18:00:47.686146 27471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } }
I20260928 18:00:47.686230 27471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 18:00:47.686269 27471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 18:00:47.686319 27471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:47.687168 27471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } }
I20260928 18:00:47.687290 27471 leader_election.cc:302] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3164b18434854863b960bd42336900b1; no voters:
I20260928 18:00:47.687484 27471 leader_election.cc:288] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 18:00:47.687541 27474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:47.687753 27471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:47.687762 27474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [term 1 LEADER]: Becoming Leader. State: Replica: 3164b18434854863b960bd42336900b1, State: Running, Role: LEADER
I20260928 18:00:47.687959 27474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } }
I20260928 18:00:47.688285 27475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3164b18434854863b960bd42336900b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } } }
I20260928 18:00:47.688381 27475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:47.688290 27476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3164b18434854863b960bd42336900b1. Latest consensus state: current_term: 1 leader_uuid: "3164b18434854863b960bd42336900b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3164b18434854863b960bd42336900b1" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 44471 } } }
I20260928 18:00:47.688514 27476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:47.688654 27482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:47.689193 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27403
I20260928 18:00:47.689283 27482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:47.689299 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:47.689478 5305 external_mini_cluster.cc:1428] Setting key ab427fbfde94e84c905d35c8e6e53349
I20260928 18:00:47.691041 27482 catalog_manager.cc:1347] Generated new cluster ID: befa52a121d54a579b10091b6f35dc5b
I20260928 18:00:47.691097 27482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:47.691903 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:47.689984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39145 (local address 127.5.46.126:44471)
0928 18:00:47.690311 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:47.690320 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:47.690354 (+ 34us) server_negotiation.cc:407] Connection header received
0928 18:00:47.690468 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:47.690475 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:47.690517 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:47.690685 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:47.691420 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:47.691423 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:47.691423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:47.691428 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:47.691473 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:47.691485 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:47.691554 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:47.691627 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 18:00:47.691672 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":75,"threads_started":1}
I20260928 18:00:47.692785 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:47.701529 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:47.700238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40401 (local address 127.5.46.126:44471)
0928 18:00:47.700326 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:47.700329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:47.700464 (+ 135us) server_negotiation.cc:407] Connection header received
0928 18:00:47.700553 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:47.700554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:47.700561 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:47.700617 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:47.701289 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:47.701290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:47.701290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 18:00:47.701295 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:47.701311 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 18:00:47.701315 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:47.701369 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:47.701433 (+ 64us) server_negotiation.cc:299] Negotiation successful
0928 18:00:47.701449 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260928 18:00:47.712051 27482 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:47.712291 27482 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:47.717695 27482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3164b18434854863b960bd42336900b1: Generated new TSK 0
I20260928 18:00:47.717939 27482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 18:00:47.724340 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27403
2026-09-28T18:00:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1349 ms)
[----------] 18 tests from AuthTokenIssuingTest (24277 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:47 dist-test-slave-z2t3 krb5kdc[27502](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:47 dist-test-slave-z2t3 krb5kdc[27502](info): set up 2 sockets
Sep 28 18:00:47 dist-test-slave-z2t3 krb5kdc[27502](info): commencing operation
krb5kdc: starting...
W20260928 18:00:48.760164 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:48 dist-test-slave-z2t3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:48.888152 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42101
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:48.971429 27515 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:48.971704 27515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:48.971747 27515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:48.971791 27515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:48.973021 27515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:48.973097 27515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:48.973121 27515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:48.973155 27515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:48.973181 27515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:48.974737 27515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:42101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:48.975026 27515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:48.975262 27515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:48.980466 27520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:48.980485 27523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:48.980469 27521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:48.980504 27515 server_base.cc:1034] running on GCE node
I20260928 18:00:48.981053 27515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:48.981352 27515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:48.982499 27515 hybrid_clock.cc:648] HybridClock initialized: now 1790618448982482 us; error 44 us; skew 500 ppm
Sep 28 18:00:48 dist-test-slave-z2t3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:48.985162 27515 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:48.985661 27515 webserver.cc:460] Webserver started at http://127.5.46.126:42797/ using document root <none> and password file <none>
I20260928 18:00:48.985913 27515 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:48.985985 27515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:48.986136 27515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:48.987236 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "318257874f224156ae44d2f2ada5a939"
format_stamp: "Formatted at 2026-09-28 18:00:48 on dist-test-slave-z2t3"
server_key: "6d7c5e7eb81d04e20cd48e7eea2d194a"
server_key_iv: "e37747270258d57728392f8c6e696638"
server_key_version: "encryptionkey@0"
I20260928 18:00:48.987607 27515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "318257874f224156ae44d2f2ada5a939"
format_stamp: "Formatted at 2026-09-28 18:00:48 on dist-test-slave-z2t3"
server_key: "6d7c5e7eb81d04e20cd48e7eea2d194a"
server_key_iv: "e37747270258d57728392f8c6e696638"
server_key_version: "encryptionkey@0"
I20260928 18:00:48.989022 27515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:48.989857 27530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:48.990057 27515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 18:00:48.990157 27515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "318257874f224156ae44d2f2ada5a939"
format_stamp: "Formatted at 2026-09-28 18:00:48 on dist-test-slave-z2t3"
server_key: "6d7c5e7eb81d04e20cd48e7eea2d194a"
server_key_iv: "e37747270258d57728392f8c6e696638"
server_key_version: "encryptionkey@0"
I20260928 18:00:48.990243 27515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:49.006742 27515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:49.009272 27515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:49.009459 27515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:49.013681 27515 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:42101
I20260928 18:00:49.013763 27582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:42101 every 8 connection(s)
I20260928 18:00:49.014127 27515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:49.015147 27583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:49.016702 27583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:49.023376 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27515
I20260928 18:00:49.023463 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:49.023650 5305 external_mini_cluster.cc:1428] Setting key 4756745492372ec826fea454c0073360
I20260928 18:00:49.027136 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.017327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42101 (local address 127.0.0.1:35504)
0928 18:00:49.017557 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.017603 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.018118 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.018419 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.018422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.018754 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.018911 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.018916 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.019986 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.019987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.020675 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.020676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.021224 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.021225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.021290 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.021953 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.021963 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.023543 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.025525 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.025531 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.025533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.025741 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.025992 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.025993 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.025993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.026088 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.026470 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.026475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.026541 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.026658 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.026915 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":106,"thread_start_us":99,"threads_started":1}
I20260928 18:00:49.027136 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.017416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35504 (local address 127.5.46.126:42101)
0928 18:00:49.017570 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.017580 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.017708 (+ 128us) server_negotiation.cc:407] Connection header received
0928 18:00:49.018286 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.018289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.018308 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.018414 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.019022 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.019909 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.020789 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.021119 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.021249 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.023628 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.023635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.023635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.023642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.025416 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.025830 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.025833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.025834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.025919 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.026156 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.026157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.026158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.026393 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.026502 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.026682 (+ 180us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.026868 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260928 18:00:49.028308 27583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.028741 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.028600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54664)
0928 18:00:49.028660 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.028716 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260928 18:00:49.028923 27532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
I20260928 18:00:49.029438 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.029325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54672)
0928 18:00:49.029381 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.029409 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 18:00:49.029559 27583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:33975: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.46.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
I20260928 18:00:49.051483 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:33975
--webserver_interface=127.5.46.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 18:00:49.078855 27583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } attempt: 1
I20260928 18:00:49.079387 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.079241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54688)
0928 18:00:49.079311 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.079356 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 18:00:49.079777 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.079662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54694)
0928 18:00:49.079715 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.079744 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 18:00:49.079900 27583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:33975: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
W20260928 18:00:49.138821 27590 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:49.139070 27590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:49.139110 27590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:49.139180 27590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:49.140415 27590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:49.140477 27590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:49.140501 27590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:49.140529 27590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:49.140553 27590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:49.142020 27590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:33975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.46.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:49.142297 27590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:49.142534 27590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:49.147543 27595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:49.147607 27590 server_base.cc:1034] running on GCE node
W20260928 18:00:49.147569 27596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:49.147569 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:49.148052 27583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } attempt: 2
I20260928 18:00:49.148145 27590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:49.148435 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.148308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54698)
0928 18:00:49.148368 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.148397 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 18:00:49.148470 27590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:49.148810 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.148695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54702)
0928 18:00:49.148755 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.148785 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260928 18:00:49.148917 27583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:33975: Network error: Client connection negotiation failed: client connection to 127.5.46.125:33975: connect: Connection refused (error 111)
I20260928 18:00:49.149644 27590 hybrid_clock.cc:648] HybridClock initialized: now 1790618449149623 us; error 33 us; skew 500 ppm
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:49.152226 27590 init.cc:377] Logged in from keytab as kudu/127.5.46.125@KRBTEST.COM (short username kudu)
I20260928 18:00:49.152725 27590 webserver.cc:460] Webserver started at http://127.5.46.125:42119/ using document root <none> and password file <none>
I20260928 18:00:49.152971 27590 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:49.153017 27590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:49.153151 27590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:49.154228 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data/instance:
uuid: "483da4b7b8b3432fa0023096b8196b71"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "48334bd85d7e4ac0a61ef432283d375e"
server_key_iv: "b64591e633a29009a278f9141e472cd2"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.154608 27590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal/instance:
uuid: "483da4b7b8b3432fa0023096b8196b71"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "48334bd85d7e4ac0a61ef432283d375e"
server_key_iv: "b64591e633a29009a278f9141e472cd2"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.156085 27590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 18:00:49.156975 27605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:49.157191 27590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:49.157300 27590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "483da4b7b8b3432fa0023096b8196b71"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "48334bd85d7e4ac0a61ef432283d375e"
server_key_iv: "b64591e633a29009a278f9141e472cd2"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.157379 27590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:49.173527 27590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:49.175601 27590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:49.175762 27590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:49.179956 27590 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.125:33975
I20260928 18:00:49.180019 27657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.125:33975 every 8 connection(s)
I20260928 18:00:49.180373 27590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
I20260928 18:00:49.181422 27658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:49.183246 27658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.186853 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27590
I20260928 18:00:49.186990 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-1/wal/instance
I20260928 18:00:49.187165 5305 external_mini_cluster.cc:1428] Setting key 621961f2775460ea8c34de1802171d74
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:49.193122 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.184057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35518 (local address 127.5.46.126:42101)
0928 18:00:49.184138 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.184140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.184351 (+ 211us) server_negotiation.cc:407] Connection header received
0928 18:00:49.184816 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.184818 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.184828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.184883 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.185468 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.185966 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.187026 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.187295 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.187389 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.189861 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.189863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.189863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.189871 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.191600 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.192087 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.192089 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.192089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.192156 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.192405 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.192406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.192407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.192567 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.192647 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.192898 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.193010 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260928 18:00:49.193379 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.184037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42101 (local address 127.0.0.1:35518)
0928 18:00:49.184227 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.184264 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.184723 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.184937 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.184940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.185194 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.185385 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.185389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.186046 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.186047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.186903 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.186905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.187417 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.187419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.187546 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.188129 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.188138 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.189763 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.191697 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.191704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.191705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.192015 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.192229 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.192229 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.192230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.192337 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.192641 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.192646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.192763 (+ 117us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.192858 (+ 95us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.193088 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":101,"threads_started":1}
I20260928 18:00:49.194247 27658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.46.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.124@KRBTEST.COM" created.
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:49.201844 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.194532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54718)
0928 18:00:49.194821 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.194832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.194880 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.195026 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.195027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.195173 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.195232 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.195234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.196163 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.196164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.196714 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.196716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.197206 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.197207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.197285 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.197663 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.197667 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.198986 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.200764 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.200764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.200765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.200930 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.201152 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.201152 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.201153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.201206 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.201526 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.201527 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.201565 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.201630 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.201731 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272}
I20260928 18:00:49.201994 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.194611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.5.46.125:33975)
0928 18:00:49.194852 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.194856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.194874 (+ 18us) server_negotiation.cc:407] Connection header received
0928 18:00:49.194944 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.194945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.194956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.195020 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.195300 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.196087 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.196786 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.197104 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.197189 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.199075 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.199083 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.199084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.199090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.200659 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.201025 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.201028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.201028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.201084 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.201278 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.201279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.201279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.201453 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.201548 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.201658 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.201782 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":190,"thread_start_us":149,"threads_started":1}
I20260928 18:00:49.202751 27658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.203107 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.202988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:44996)
0928 18:00:49.203036 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.203083 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 18:00:49.203249 27606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
I20260928 18:00:49.203692 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.203598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45004)
0928 18:00:49.203647 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.203670 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 18:00:49.203796 27658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:44535: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
I20260928 18:00:49.215695 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:44535
--webserver_interface=127.5.46.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 18:00:49.240047 27583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } attempt: 3
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:49.248104 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.240532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.5.46.125:33975)
0928 18:00:49.240604 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.240607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.240617 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:49.240647 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.240648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.240707 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.240773 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.241121 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.241717 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.242465 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.242739 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.242817 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.245174 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.245176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.245176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.245192 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.246811 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.247235 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.247236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.247236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.247297 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.247517 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.247518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.247519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.247644 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.247724 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.247913 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.248001 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 18:00:49.248104 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.240431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54728)
0928 18:00:49.240488 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.240500 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.240550 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.240794 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.240795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.240966 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.241036 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.241038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.241804 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.241806 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.242346 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.242347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.242815 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.242816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.242904 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.243473 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.243478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.245060 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.246939 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.246941 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.246941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.247132 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.247379 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.247380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.247380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.247445 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.247717 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.247718 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.247772 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.247883 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.248001 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 18:00:49.248639 27583 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.249028 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.248921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45010)
0928 18:00:49.248975 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.249009 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 18:00:49.249466 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.249369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45026)
0928 18:00:49.249418 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.249442 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 18:00:49.249558 27583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:44535: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
I20260928 18:00:49.253062 27658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } attempt: 1
I20260928 18:00:49.253525 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.253397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45042)
0928 18:00:49.253456 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.253487 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 18:00:49.253885 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.253789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45046)
0928 18:00:49.253838 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.253861 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 18:00:49.253980 27658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:44535: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
I20260928 18:00:49.280768 27583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } attempt: 1
I20260928 18:00:49.281282 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.281136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45050)
0928 18:00:49.281212 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.281250 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 18:00:49.281649 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.281546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45066)
0928 18:00:49.281596 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.281622 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 18:00:49.281751 27583 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:44535: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
W20260928 18:00:49.304931 27665 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:49.305209 27665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:49.305240 27665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:49.305286 27665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:49.306497 27665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:49.306574 27665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:49.306602 27665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:49.306634 27665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:49.306661 27665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:49.308296 27665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:42101,127.5.46.125:33975,127.5.46.124:44535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:44535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.46.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:49.308598 27665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:49.308851 27665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:49.314047 27671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:49.314204 27665 server_base.cc:1034] running on GCE node
W20260928 18:00:49.314006 27670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:49.314015 27673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:49.314642 27665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:49.314985 27665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:49.316193 27665 hybrid_clock.cc:648] HybridClock initialized: now 1790618449316163 us; error 48 us; skew 500 ppm
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:49.318792 27665 init.cc:377] Logged in from keytab as kudu/127.5.46.124@KRBTEST.COM (short username kudu)
I20260928 18:00:49.319247 27665 webserver.cc:460] Webserver started at http://127.5.46.124:40531/ using document root <none> and password file <none>
I20260928 18:00:49.319481 27665 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:49.319551 27665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:49.319717 27665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:49.320912 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data/instance:
uuid: "291468112357400292d06f1096818b03"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "e61cc0e0482969e1115cf3f2b8060b78"
server_key_iv: "321ef489cc25d446ccff88df312dbdf5"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.321296 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal/instance:
uuid: "291468112357400292d06f1096818b03"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "e61cc0e0482969e1115cf3f2b8060b78"
server_key_iv: "321ef489cc25d446ccff88df312dbdf5"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.322129 27658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } attempt: 2
I20260928 18:00:49.322602 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.322452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45070)
0928 18:00:49.322523 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.322564 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 18:00:49.322937 27665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 18:00:49.322999 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.322913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45074)
0928 18:00:49.322956 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.322982 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 18:00:49.323101 27658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:44535: Network error: Client connection negotiation failed: client connection to 127.5.46.124:44535: connect: Connection refused (error 111)
I20260928 18:00:49.323743 27680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:49.323978 27665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 18:00:49.324079 27665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "291468112357400292d06f1096818b03"
format_stamp: "Formatted at 2026-09-28 18:00:49 on dist-test-slave-z2t3"
server_key: "e61cc0e0482969e1115cf3f2b8060b78"
server_key_iv: "321ef489cc25d446ccff88df312dbdf5"
server_key_version: "encryptionkey@0"
I20260928 18:00:49.324164 27665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:49.347968 27665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:49.350121 27665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:49.350306 27665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:49.354518 27665 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.124:44535
I20260928 18:00:49.354553 27732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.124:44535 every 8 connection(s)
I20260928 18:00:49.354925 27665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
I20260928 18:00:49.356093 27733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:49.356920 27583 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } attempt: 2
I20260928 18:00:49.358106 27733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.361018 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27665
I20260928 18:00:49.361163 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790618177091767-5305-0/minicluster-data/master-2/wal/instance
I20260928 18:00:49.361357 5305 external_mini_cluster.cc:1428] Setting key cc36eaca620343cb3b76d9d8922c2152
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:49.367869 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.358790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.5.46.126:42101)
0928 18:00:49.358846 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.358848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.359008 (+ 160us) server_negotiation.cc:407] Connection header received
0928 18:00:49.359041 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.359042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.359052 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.359122 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.359554 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.360251 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.361485 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.361746 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.361851 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.364643 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.364644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.364645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.364652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.366472 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.366838 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.366839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.366839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.366903 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.367153 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.367154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.367154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.367386 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.367466 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.367651 (+ 185us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.367752 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":33}
I20260928 18:00:49.367976 27585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.357242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45082)
0928 18:00:49.357333 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.357346 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.357412 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.358307 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.358309 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.358501 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.358547 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.358549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.360116 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.360118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.360976 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.360979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.361746 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.361747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.361803 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.362340 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.362345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.363874 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.366171 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.366173 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.366174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.366828 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.367109 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.367110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.367110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.367173 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.367533 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.367534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.367577 (+ 43us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.367641 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.367817 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":70}
I20260928 18:00:49.368095 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.358725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:42101 (local address 127.0.0.1:35520)
0928 18:00:49.358892 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.358932 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.359001 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.359134 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.359136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.359374 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.359469 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.359474 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.360333 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.360334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.361400 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.361403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.362058 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.362059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.362144 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.362554 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.362562 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.364567 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.366547 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.366553 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.366556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.366755 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.366970 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.366971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.366972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.367090 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.367469 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.367474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.367536 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.367629 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.367776 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":113,"threads_started":1}
I20260928 18:00:49.368341 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.357459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.5.46.124:44535)
0928 18:00:49.357655 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.357667 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.357693 (+ 26us) server_negotiation.cc:407] Connection header received
0928 18:00:49.358100 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.358107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.358155 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.358311 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.358661 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.360004 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.361069 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.361640 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.361789 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.363947 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.363954 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.363955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.363962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.366089 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.366929 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.366933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.366934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.367035 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.367223 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.367224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.367225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.367471 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.367551 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.367885 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.368212 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:49.368602 27733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.370235 27583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: Bootstrap starting.
I20260928 18:00:49.371207 27583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:49.371526 27583 log.cc:824] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:49.372430 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.361916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.5.46.124:44535)
0928 18:00:49.362041 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.362043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.362052 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:49.362113 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.362113 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.362125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.362191 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.363636 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.364359 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.365377 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.365681 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.365754 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.369044 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.369045 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.369046 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.369052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.371090 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.371481 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.371482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.371483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.371545 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.371750 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.371751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.371751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.371917 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.372059 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.372130 (+ 71us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.372220 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260928 18:00:49.372792 27583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: No bootstrap required, opened a new log
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:49.375185 27583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.375337 27583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318257874f224156ae44d2f2ada5a939, State: Initialized, Role: FOLLOWER
I20260928 18:00:49.375416 27583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.375712 27741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.375787 27583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:49.375794 27741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.377152 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.369025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.5.46.125:33975)
0928 18:00:49.369067 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.369069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.369171 (+ 102us) server_negotiation.cc:407] Connection header received
0928 18:00:49.369206 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.369207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.369220 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.369283 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.369565 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.370254 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.371249 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.371549 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.371595 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.373722 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.373723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.373723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.373731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.375372 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.375771 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.375772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.375772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.375880 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.376392 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.376393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.376394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.376539 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.376588 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.376970 (+ 382us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.377061 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 18:00:49.377583 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.368838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:33975 (local address 127.0.0.1:54734)
0928 18:00:49.369094 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.369104 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.369158 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.369284 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.369285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.369419 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.369469 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.369470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.370356 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.370356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.371140 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.371143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.371677 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.371678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.371754 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.372201 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.372206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.373623 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.375469 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.375470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.375470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.375684 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.376243 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.376244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.376244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.376304 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.376638 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.376639 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.376690 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.377373 (+ 683us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.377484 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":88,"threads_started":1}
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
W20260928 18:00:49.377969 27752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:49.378021 27752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:49.378196 27733 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:49.378414 27515 master_runner.cc:186] Error getting master registration for 127.5.46.125:33975: OK, instance_id { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" instance_seqno: 1790618449178766 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 18:00:49.381484 27515 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 18:00:49.381717 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.373202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.5.46.126:42101)
0928 18:00:49.373241 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.373243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.373283 (+ 40us) server_negotiation.cc:407] Connection header received
0928 18:00:49.373374 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.373375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.373385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.373413 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.373903 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.374476 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.375345 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.375648 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.375714 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.378662 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.378664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.378664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.378672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.380432 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.380835 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.380836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.380836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.380893 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.381082 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.381083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.381083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.381199 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.381322 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.381544 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.381625 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:49.387035 27736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.378430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45102)
0928 18:00:49.378517 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.378528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.378570 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.378735 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.378736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.379252 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.379297 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.379301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.380560 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.380561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.381329 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.381331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.381899 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.381900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.381957 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.382488 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.382493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.383991 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.385834 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.385834 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.385835 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.386042 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.386229 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.386230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.386230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.386278 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.386631 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.386632 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.386668 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.386738 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.386858 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:49.387102 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.378523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45102 (local address 127.5.46.124:44535)
0928 18:00:49.378563 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.378565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.378574 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:49.378644 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.378645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.378653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.378681 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.379391 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.380482 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.381409 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.381725 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.381770 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.384088 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.384090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.384091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.384097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.385732 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.386118 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.386119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.386119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.386167 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.386341 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.386342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.386342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.386538 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.386587 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.386794 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.386921 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":21}
I20260928 18:00:49.388660 27733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: Bootstrap starting.
I20260928 18:00:49.389312 27733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:49.389618 27733 log.cc:824] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:49.390435 27733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: No bootstrap required, opened a new log
I20260928 18:00:49.392355 27733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.392607 27733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 291468112357400292d06f1096818b03, State: Initialized, Role: FOLLOWER
I20260928 18:00:49.392736 27733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.393097 27733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:49.393471 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.382663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.5.46.125:33975)
0928 18:00:49.382704 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.382706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.382714 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:49.382773 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.382774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.382783 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.382845 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.383332 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.383967 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.384832 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.385156 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.385261 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.390111 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.390113 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.390113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.390123 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.392040 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.392495 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.392497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.392497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.392561 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.392788 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.392789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.392789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.392952 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.393112 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.393175 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.393258 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 18:00:49.394742 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.395330 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:49.396373 27557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "318257874f224156ae44d2f2ada5a939"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35536
I20260928 18:00:49.396529 27557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 18:00:49.396581 27557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:49.397321 27557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.397557 27557 leader_election.cc:288] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [CANDIDATE]: Term 1 election: Requested vote from peers 483da4b7b8b3432fa0023096b8196b71 (127.5.46.125:33975), 291468112357400292d06f1096818b03 (127.5.46.124:44535)
I20260928 18:00:49.394922 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 18:00:49.395370 27766 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:49.397799 27766 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:49.397819 27707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "318257874f224156ae44d2f2ada5a939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "291468112357400292d06f1096818b03"
I20260928 18:00:49.397924 27707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:49.398798 27707 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318257874f224156ae44d2f2ada5a939 in term 1.
I20260928 18:00:49.399775 27531 leader_election.cc:302] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 291468112357400292d06f1096818b03, 318257874f224156ae44d2f2ada5a939; no voters:
I20260928 18:00:49.399952 27741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:49.397701 27665 master_runner.cc:186] Error getting master registration for 127.5.46.125:33975: OK, instance_id { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" instance_seqno: 1790618449178766 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 18:00:49.400151 27741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [term 1 LEADER]: Becoming Leader. State: Replica: 318257874f224156ae44d2f2ada5a939, State: Running, Role: LEADER
I20260928 18:00:49.400199 27665 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 18:00:49.400260 27741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.400687 27741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318257874f224156ae44d2f2ada5a939. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.400771 27741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:49.410295 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.399690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54760 (local address 127.5.46.125:33975)
0928 18:00:49.399728 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.399730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.400303 (+ 573us) server_negotiation.cc:407] Connection header received
0928 18:00:49.400452 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.400454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.400464 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.400496 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.401572 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.402346 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.403819 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.404152 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.404192 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.406655 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.406656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.406656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.406664 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.408445 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.409070 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.409071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.409072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.409149 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.409483 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.409485 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.409485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.409641 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.409720 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.410087 (+ 367us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.410175 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 18:00:49.412015 27777 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:49.412252 27777 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:49.413642 27586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.399707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35542 (local address 127.5.46.126:42101)
0928 18:00:49.399968 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.399970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.399978 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:49.399999 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.400000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.400010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.400067 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.402691 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.403407 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.405284 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.405600 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.405649 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.408629 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.408631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.408631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.408640 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.410377 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.412456 (+ 2079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.412458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.412458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.412524 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.412996 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.412997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.412998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.413143 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.413228 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.413448 (+ 220us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.413537 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260928 18:00:49.414224 27658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } attempt: 3
I20260928 18:00:49.414870 27707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [term 1 FOLLOWER]: Refusing update from remote peer 318257874f224156ae44d2f2ada5a939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:49.415208 27741 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:49.416636 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318257874f224156ae44d2f2ada5a939. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.416736 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.419106 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.399576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.5.46.124:44535)
0928 18:00:49.399616 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.399618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.400368 (+ 750us) server_negotiation.cc:407] Connection header received
0928 18:00:49.400539 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.400540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.400550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.400592 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.401917 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.402601 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.405729 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.406078 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.406126 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.411744 (+ 5618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.411746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.411746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.411757 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.413509 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.414224 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.414226 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.414226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.414292 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.417933 (+ 3641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.417934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.417934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.418087 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.418178 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.418914 (+ 736us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.418997 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 28 18:00:49 dist-test-slave-z2t3 krb5kdc[27502](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618449, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:49.423069 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.414467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:44535 (local address 127.0.0.1:45118)
0928 18:00:49.414557 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:49.414569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:49.414609 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:49.414863 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:49.414864 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:49.415021 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:49.415071 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.415073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.416108 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.416110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.416869 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:49.416872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.417437 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.417438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:49.417504 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.417922 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:49.417942 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:49.420032 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:49.421962 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.421963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.421963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.422192 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.422431 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:49.422432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:49.422432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:49.422492 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:49.422748 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:49.422749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:49.422800 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 18:00:49.422868 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 18:00:49.422968 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260928 18:00:49.423069 27735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:49.414696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45118 (local address 127.5.46.124:44535)
0928 18:00:49.414734 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.414737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.414743 (+ 6us) server_negotiation.cc:407] Connection header received
0928 18:00:49.414761 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.414762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.414771 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.414816 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.415151 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.415914 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.417019 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:49.417359 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:49.417424 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:49.420145 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:49.420147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:49.420148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:49.420156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:49.421881 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.422270 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.422272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.422272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.422331 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:49.422562 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:49.422563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:49.422563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:49.422689 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:49.422748 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:49.422903 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 18:00:49.422979 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 18:00:49.425143 27658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: Bootstrap starting.
I20260928 18:00:49.425952 27658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:49.426193 27755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.426278 27755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.426288 27658 log.cc:824] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:49.427150 27775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.427415 27775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:49.427487 27779 mvcc.cc:205] Tried to move back new op lower bound from 7334373168800686080 to 7334373168744255488. Current Snapshot: MvccSnapshot[applied={T|T < 7334373168800686080}]
I20260928 18:00:49.428227 27658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: No bootstrap required, opened a new log
I20260928 18:00:49.430303 27658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.430450 27658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 483da4b7b8b3432fa0023096b8196b71, State: Initialized, Role: FOLLOWER
I20260928 18:00:49.430534 27658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } }
I20260928 18:00:49.430871 27658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:49.430954 27632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "318257874f224156ae44d2f2ada5a939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "483da4b7b8b3432fa0023096b8196b71"
I20260928 18:00:49.431061 27632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:49.432024 27632 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318257874f224156ae44d2f2ada5a939 in term 1.
I20260928 18:00:49.432466 27631 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [term 1 FOLLOWER]: Refusing update from remote peer 318257874f224156ae44d2f2ada5a939: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:49.433089 27775 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [LEADER]: Connected to new peer: Peer: permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 18:00:49.436233 27796 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:49.436302 27796 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:49.436528 27785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.436609 27785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.436658 27785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 318257874f224156ae44d2f2ada5a939. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.436712 27785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.436748 27785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.436807 27785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:49.437393 27798 mvcc.cc:205] Tried to move back new op lower bound from 7334373168800686080 to 7334373168744255488. Current Snapshot: MvccSnapshot[applied={T|T < 7334373168800686080}]
I20260928 18:00:49.438781 27777 catalog_manager.cc:1347] Generated new cluster ID: ac0baa55361e4bedbb4de97dbea33690
I20260928 18:00:49.438850 27777 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:49.452275 27782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "318257874f224156ae44d2f2ada5a939" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "318257874f224156ae44d2f2ada5a939" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 42101 } } peers { permanent_uuid: "483da4b7b8b3432fa0023096b8196b71" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 33975 } } peers { permanent_uuid: "291468112357400292d06f1096818b03" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 44535 } } }
I20260928 18:00:49.452430 27782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:49.455111 27777 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:49.456113 27777 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:49.465730 27777 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 318257874f224156ae44d2f2ada5a939: Generated new TSK 0
I20260928 18:00:49.465976 27777 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 18:00:49.496477 27586 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.494825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.5.46.126:42101)
0928 18:00:49.494873 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.494875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.495757 (+ 882us) server_negotiation.cc:407] Connection header received
0928 18:00:49.495892 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.495893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.495905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.495936 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.496404 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35558: BlockingRecv error: recv got EOF from 127.0.0.1:35558 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 18:00:49.497251 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.495186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.5.46.125:33975)
0928 18:00:49.495228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.495230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.496726 (+ 1496us) server_negotiation.cc:407] Connection header received
0928 18:00:49.496828 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.496830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.496842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.496872 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.497186 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54776: BlockingRecv error: recv got EOF from 127.0.0.1:54776 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 18:00:49.497993 27735 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.495501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.5.46.124:44535)
0928 18:00:49.495587 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.495589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.497494 (+ 1905us) server_negotiation.cc:407] Connection header received
0928 18:00:49.497593 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.497594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.497605 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.497635 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.497935 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45124: BlockingRecv error: recv got EOF from 127.0.0.1:45124 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260928 18:00:49.500123 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.499287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.5.46.125:33975)
0928 18:00:49.499336 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.499338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.499554 (+ 216us) server_negotiation.cc:407] Connection header received
0928 18:00:49.499669 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.499670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.499689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.499719 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.500077 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54792: BlockingRecv error: recv got EOF from 127.0.0.1:54792 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 18:00:49.503296 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.502513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.5.46.125:33975)
0928 18:00:49.502547 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.502549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.502783 (+ 234us) server_negotiation.cc:407] Connection header received
0928 18:00:49.502883 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.502884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.502895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.502925 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.503245 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54798: BlockingRecv error: recv got EOF from 127.0.0.1:54798 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 18:00:49.507345 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.506506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.5.46.125:33975)
0928 18:00:49.506544 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.506546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.506792 (+ 246us) server_negotiation.cc:407] Connection header received
0928 18:00:49.506898 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.506899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.506911 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.506941 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.507281 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54800: BlockingRecv error: recv got EOF from 127.0.0.1:54800 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 18:00:49.513736 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.512738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.5.46.125:33975)
0928 18:00:49.512783 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.512785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.513069 (+ 284us) server_negotiation.cc:407] Connection header received
0928 18:00:49.513198 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.513199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.513212 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.513253 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.513673 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54810: BlockingRecv error: recv got EOF from 127.0.0.1:54810 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 18:00:49.524271 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.523237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.5.46.125:33975)
0928 18:00:49.523283 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.523285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.523581 (+ 296us) server_negotiation.cc:407] Connection header received
0928 18:00:49.523712 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.523713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.523727 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.523759 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.524206 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54818: BlockingRecv error: recv got EOF from 127.0.0.1:54818 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 18:00:49.542805 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.541788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.5.46.125:33975)
0928 18:00:49.541837 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.541839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.542123 (+ 284us) server_negotiation.cc:407] Connection header received
0928 18:00:49.542239 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.542240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.542253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.542285 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.542744 (+ 459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54828: BlockingRecv error: recv got EOF from 127.0.0.1:54828 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 18:00:49.577236 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.576295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.5.46.125:33975)
0928 18:00:49.576349 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.576352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.576514 (+ 162us) server_negotiation.cc:407] Connection header received
0928 18:00:49.576594 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.576596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.576609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.576687 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.577177 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54844: BlockingRecv error: recv got EOF from 127.0.0.1:54844 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 18:00:49.643546 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.642832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54850 (local address 127.5.46.125:33975)
0928 18:00:49.642889 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.642891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.642907 (+ 16us) server_negotiation.cc:407] Connection header received
0928 18:00:49.643007 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.643008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.643026 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.643092 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.643477 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54850: BlockingRecv error: recv got EOF from 127.0.0.1:54850 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260928 18:00:49.773805 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:49.772953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.5.46.125:33975)
0928 18:00:49.773045 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:49.773048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:49.773147 (+ 99us) server_negotiation.cc:407] Connection header received
0928 18:00:49.773264 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:49.773265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:49.773278 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:49.773358 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:49.773714 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54852: BlockingRecv error: recv got EOF from 127.0.0.1:54852 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260928 18:00:50.032162 27662 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:50.031263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.5.46.125:33975)
0928 18:00:50.031365 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.031368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.031471 (+ 103us) server_negotiation.cc:407] Connection header received
0928 18:00:50.031572 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.031573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.031586 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.031666 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:50.032111 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54866: BlockingRecv error: recv got EOF from 127.0.0.1:54866 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260928 18:00:50.398253 27766 catalog_manager.cc:1259] Loaded cluster ID: ac0baa55361e4bedbb4de97dbea33690
I20260928 18:00:50.398335 27766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: loading cluster ID for follower catalog manager: success
I20260928 18:00:50.399290 27766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: acquiring CA information for follower catalog manager: success
I20260928 18:00:50.399616 27766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 291468112357400292d06f1096818b03: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 18:00:50.436793 27796 catalog_manager.cc:1259] Loaded cluster ID: ac0baa55361e4bedbb4de97dbea33690
I20260928 18:00:50.436878 27796 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: loading cluster ID for follower catalog manager: success
I20260928 18:00:50.437649 27796 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: acquiring CA information for follower catalog manager: success
I20260928 18:00:50.437961 27796 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 483da4b7b8b3432fa0023096b8196b71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 18:00:50.548561 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:50.545642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.5.46.125:33975)
0928 18:00:50.545767 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.545768 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.545779 (+ 11us) server_negotiation.cc:407] Connection header received
0928 18:00:50.545862 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.545863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.545913 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.545995 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:50.546174 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.546764 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.547590 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.547821 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.547903 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:50.548120 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:50.548260 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:50.548329 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:50.548401 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 18:00:50.548421 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260928 18:00:50.554101 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:50.551018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.5.46.124:44535)
0928 18:00:50.551162 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.551165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.551173 (+ 8us) server_negotiation.cc:407] Connection header received
0928 18:00:50.551213 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.551213 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.551250 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.551336 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:50.551514 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.552095 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.552901 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.553104 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.553165 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:50.553495 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:50.553616 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:50.553702 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:50.553902 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 18:00:50.553968 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260928 18:00:50.559371 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:50.556270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45136 (local address 127.5.46.124:44535)
0928 18:00:50.556344 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.556348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.556494 (+ 146us) server_negotiation.cc:407] Connection header received
0928 18:00:50.556561 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.556562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.556573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.556723 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:50.556766 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.557533 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.558569 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.558843 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.558910 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:50.559075 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:50.559116 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:50.559166 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:50.559248 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 18:00:50.559274 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 18:00:50.559788 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:50.556424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.5.46.125:33975)
0928 18:00:50.556478 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.556480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.556577 (+ 97us) server_negotiation.cc:407] Connection header received
0928 18:00:50.556673 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.556674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.556688 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.556718 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:50.556938 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.557699 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.558786 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.559108 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.559181 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:50.559355 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:50.559388 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:50.559415 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:50.559658 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 18:00:50.559688 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 18:00:50.559965 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:50.556185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.5.46.126:42101)
0928 18:00:50.556355 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:50.556357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:50.556386 (+ 29us) server_negotiation.cc:407] Connection header received
0928 18:00:50.556514 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:50.556515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:50.556576 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:50.556642 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:50.556890 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.557483 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.558745 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:50.559079 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:50.559153 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:50.559355 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:50.559493 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:50.559535 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:50.559801 (+ 266us) server_negotiation.cc:299] Negotiation successful
0928 18:00:50.559841 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":57,"threads_started":1}
I20260928 18:00:50.562109 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27515
I20260928 18:00:50.567428 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27590
I20260928 18:00:50.572702 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27665
2026-09-28T18:00:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2850 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 18:00:50 dist-test-slave-z2t3 krb5kdc[27884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 18:00:50 dist-test-slave-z2t3 krb5kdc[27884](info): set up 2 sockets
Sep 28 18:00:50 dist-test-slave-z2t3 krb5kdc[27884](info): commencing operation
krb5kdc: starting...
W20260928 18:00:51.612349 5305 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/test-user.keytab.
Sep 28 18:00:51 dist-test-slave-z2t3 krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T18:00:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T18:00:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.46.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
Entry for principal kudu/127.5.46.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab.
I20260928 18:00:51.736388 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46663
--webserver_interface=127.5.46.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 18:00:51.820202 27897 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:51.820439 27897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:51.820457 27897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:51.820479 27897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:51.821720 27897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:51.821770 27897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:51.821785 27897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:51.821805 27897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:51.821817 27897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:51.823369 27897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.126.keytab
--principal=kudu/127.5.46.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.126:46663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.46.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:51.823594 27897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:51.823786 27897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:51.829003 27903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:51.829003 27905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:51.829026 27902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:51.829324 27897 server_base.cc:1034] running on GCE node
I20260928 18:00:51.829670 27897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:51.830067 27897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:51.831240 27897 hybrid_clock.cc:648] HybridClock initialized: now 1790618451831229 us; error 54 us; skew 500 ppm
Sep 28 18:00:51 dist-test-slave-z2t3 krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:51.834221 27897 init.cc:377] Logged in from keytab as kudu/127.5.46.126@KRBTEST.COM (short username kudu)
I20260928 18:00:51.834822 27897 webserver.cc:460] Webserver started at http://127.5.46.126:40369/ using document root <none> and password file <none>
I20260928 18:00:51.835103 27897 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:51.835176 27897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:51.835335 27897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:51.836524 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data/instance:
uuid: "37691a8e77714144887a617d04d3cdc7"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "96da86dab77b4dc994f0350763c18b82"
server_key_iv: "e9fc578d7f8be987f60baeecb465e554"
server_key_version: "encryptionkey@0"
I20260928 18:00:51.836898 27897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance:
uuid: "37691a8e77714144887a617d04d3cdc7"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "96da86dab77b4dc994f0350763c18b82"
server_key_iv: "e9fc578d7f8be987f60baeecb465e554"
server_key_version: "encryptionkey@0"
I20260928 18:00:51.838351 27897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 18:00:51.839201 27912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:51.839448 27897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:51.839563 27897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal
uuid: "37691a8e77714144887a617d04d3cdc7"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "96da86dab77b4dc994f0350763c18b82"
server_key_iv: "e9fc578d7f8be987f60baeecb465e554"
server_key_version: "encryptionkey@0"
I20260928 18:00:51.839643 27897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:51.856849 27897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:51.859032 27897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:51.859216 27897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:51.863718 27897 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.126:46663
I20260928 18:00:51.863791 27964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.126:46663 every 8 connection(s)
I20260928 18:00:51.864213 27897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/data/info.pb
I20260928 18:00:51.865260 27965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:51.866916 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:51.871721 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27897
I20260928 18:00:51.871955 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-0/wal/instance
I20260928 18:00:51.872225 5305 external_mini_cluster.cc:1428] Setting key bcf0acf09d5167e3beda1f2d49eba1a8
Sep 28 18:00:51 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:51.878207 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.867656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46663 (local address 127.0.0.1:48628)
0928 18:00:51.867954 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.868007 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:51.868587 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:51.868975 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:51.868978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:51.869301 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:51.869476 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:51.869481 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:51.870581 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:51.870583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:51.871306 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:51.871308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:51.871904 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:51.871905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:51.871973 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:51.872666 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:51.872675 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:51.874178 (+ 1503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:51.876594 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:51.876601 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:51.876603 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:51.876814 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:51.877137 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:51.877138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:51.877138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:51.877223 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:51.877740 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:51.877745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:51.877804 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 18:00:51.877891 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 18:00:51.878030 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":127,"threads_started":1}
I20260928 18:00:51.878306 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.867753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.5.46.126:46663)
0928 18:00:51.867966 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:51.867977 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:51.868133 (+ 156us) server_negotiation.cc:407] Connection header received
0928 18:00:51.868815 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:51.868819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:51.868846 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:51.868958 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:51.869597 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:51.870449 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:51.871420 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:51.871747 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:51.871916 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:51.874296 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:51.874306 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:51.874306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:51.874315 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:51.876483 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:51.876916 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:51.876920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:51.876920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:51.877051 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:51.877296 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:51.877297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:51.877297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:51.877644 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:51.877765 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:51.877964 (+ 199us) server_negotiation.cc:299] Negotiation successful
0928 18:00:51.878128 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260928 18:00:51.879217 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:51.879608 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.879490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57066)
0928 18:00:51.879540 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.879588 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 18:00:51.879781 27916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
I20260928 18:00:51.880262 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.880171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57080)
0928 18:00:51.880221 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.880246 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 18:00:51.880496 27965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:41279: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.5.46.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
Entry for principal kudu/127.5.46.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab.
I20260928 18:00:51.899487 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:41279
--webserver_interface=127.5.46.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 18:00:51.929744 27965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } attempt: 1
I20260928 18:00:51.930248 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.930109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57088)
0928 18:00:51.930176 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.930218 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 18:00:51.930629 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.930526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57094)
0928 18:00:51.930576 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.930601 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 18:00:51.930721 27965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:41279: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
W20260928 18:00:51.983215 27972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:51.983489 27972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:51.983520 27972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:51.983563 27972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:51.984831 27972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:51.984905 27972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:51.984928 27972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:51.984949 27972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:51.984980 27972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:51.986509 27972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.125.keytab
--principal=kudu/127.5.46.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.125:41279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.46.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:51.986817 27972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:51.987057 27972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:51.992115 27980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:51.992162 27972 server_base.cc:1034] running on GCE node
W20260928 18:00:51.992167 27978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:51.992117 27977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:51.992573 27972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:51.992883 27972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:51.994046 27972 hybrid_clock.cc:648] HybridClock initialized: now 1790618451994030 us; error 51 us; skew 500 ppm
Sep 28 18:00:51 dist-test-slave-z2t3 krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:51.996544 27972 init.cc:377] Logged in from keytab as kudu/127.5.46.125@KRBTEST.COM (short username kudu)
I20260928 18:00:51.997013 27972 webserver.cc:460] Webserver started at http://127.5.46.125:35273/ using document root <none> and password file <none>
I20260928 18:00:51.997251 27972 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:51.997323 27972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:51.997480 27972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:51.998551 27972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data/instance:
uuid: "8886a4d9a23746baa34581be2f536696"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "909a0d87e2b76bdb745b4eec1e326406"
server_key_iv: "5fea7df27e0cec01d0d69cd4fa811daa"
server_key_version: "encryptionkey@0"
I20260928 18:00:51.998867 27965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } attempt: 2
I20260928 18:00:51.998998 27972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal/instance:
uuid: "8886a4d9a23746baa34581be2f536696"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "909a0d87e2b76bdb745b4eec1e326406"
server_key_iv: "5fea7df27e0cec01d0d69cd4fa811daa"
server_key_version: "encryptionkey@0"
I20260928 18:00:51.999269 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.999129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57096)
0928 18:00:51.999203 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.999239 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 18:00:51.999733 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:51.999632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57106)
0928 18:00:51.999685 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:51.999710 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 18:00:51.999881 27965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.125:41279: Network error: Client connection negotiation failed: client connection to 127.5.46.125:41279: connect: Connection refused (error 111)
I20260928 18:00:52.000747 27972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 18:00:52.001557 27987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:52.001806 27972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 18:00:52.001888 27972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal
uuid: "8886a4d9a23746baa34581be2f536696"
format_stamp: "Formatted at 2026-09-28 18:00:51 on dist-test-slave-z2t3"
server_key: "909a0d87e2b76bdb745b4eec1e326406"
server_key_iv: "5fea7df27e0cec01d0d69cd4fa811daa"
server_key_version: "encryptionkey@0"
I20260928 18:00:52.001979 27972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:52.011042 27972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:52.013072 27972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:52.013247 27972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:52.017308 27972 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.125:41279
I20260928 18:00:52.017366 28039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.125:41279 every 8 connection(s)
I20260928 18:00:52.017742 27972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/data/info.pb
I20260928 18:00:52.018710 28040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:52.020372 28040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:52.024439 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 27972
I20260928 18:00:52.024566 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-1/wal/instance
I20260928 18:00:52.024748 5305 external_mini_cluster.cc:1428] Setting key bab027adc89d41f15e7164c634184e2c
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:52.030731 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.021118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48634 (local address 127.5.46.126:46663)
0928 18:00:52.021177 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.021179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.021419 (+ 240us) server_negotiation.cc:407] Connection header received
0928 18:00:52.021946 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.021947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.021964 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.022017 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.022666 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.023145 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.024212 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.024422 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.024503 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.027365 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.027367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.027367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.027377 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.029218 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.029658 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.029659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.029659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.029727 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.029970 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.029971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.029972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.030135 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.030215 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.030488 (+ 273us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.030590 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 18:00:52.030745 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.021086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46663 (local address 127.0.0.1:48634)
0928 18:00:52.021291 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.021332 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.021835 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.022092 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.022095 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.022351 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.022565 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.022569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.023256 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.023258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.024124 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.024126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.024528 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.024530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.024694 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.025302 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.025314 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.027234 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.029314 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.029333 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.029335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.029573 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.029800 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.029801 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.029801 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.029904 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.030208 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.030212 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.030352 (+ 140us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.030438 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.030567 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":101,"threads_started":1}
I20260928 18:00:52.031440 28040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.46.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.46.124@KRBTEST.COM" created.
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:52.040143 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.031735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57108)
0928 18:00:52.031789 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.031801 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.031903 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.032117 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.032118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.032263 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.032312 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.032314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.033190 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.033191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.033693 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.033695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.034174 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.034174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.034239 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.034705 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.034711 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.036878 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.038897 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.038898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.038899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.039131 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.039341 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.039342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.039343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.039413 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.039688 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.039689 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.039743 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.039821 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.039978 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 18:00:52.040235 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.031807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57108 (local address 127.5.46.125:41279)
0928 18:00:52.031982 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.031991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.032008 (+ 17us) server_negotiation.cc:407] Connection header received
0928 18:00:52.032024 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.032026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.032036 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.032117 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.032392 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.033070 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.033776 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.034096 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.034180 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.036998 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.037011 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.037011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.037020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.038814 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.039214 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.039217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.039217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.039270 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.039473 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.039473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.039474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.039620 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.039683 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.039919 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.040054 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":145,"thread_start_us":109,"threads_started":1}
I20260928 18:00:52.040930 28040 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:52.041344 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.041180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40188)
0928 18:00:52.041234 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.041294 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 18:00:52.041515 27990 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
I20260928 18:00:52.042043 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.041922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40192)
0928 18:00:52.041975 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.042010 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 18:00:52.042165 28040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:40983: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
Entry for principal kudu/127.5.46.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab.
I20260928 18:00:52.055943 5305 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskO6aDtj/build/release/bin/kudu
/tmp/dist-test-taskO6aDtj/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:40983
--webserver_interface=127.5.46.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.46.84:45555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 18:00:52.091066 27965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } attempt: 3
I20260928 18:00:52.091413 28040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } attempt: 1
I20260928 18:00:52.091907 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.091742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40196)
0928 18:00:52.091809 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.091867 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 18:00:52.092617 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.092513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40212)
0928 18:00:52.092558 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.092587 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 18:00:52.092747 28040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:40983: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:52.099099 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.091555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57118)
0928 18:00:52.091617 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.091631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.091676 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.091893 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.091895 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.092088 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.092207 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.092209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.093239 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.093241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.093798 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.093800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.094337 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.094338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.094409 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.094912 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.094917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.096389 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.098151 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.098152 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.098153 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.098338 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.098538 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.098538 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.098539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.098591 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.098855 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.098856 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.098896 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.098948 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.099028 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 18:00:52.099172 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.091627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.5.46.125:41279)
0928 18:00:52.091677 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.091679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.091691 (+ 12us) server_negotiation.cc:407] Connection header received
0928 18:00:52.091727 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.091728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.091740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.091827 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.092308 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.093122 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.093905 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.094237 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.094316 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.096495 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.096497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.096498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.096506 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.098059 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.098408 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.098409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.098410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.098472 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.098651 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.098651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.098652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.098792 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.098859 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.098995 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.099079 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 18:00:52.099545 27965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:52.099967 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.099839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40228)
0928 18:00:52.099906 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.099935 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260928 18:00:52.100339 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.100270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40240)
0928 18:00:52.100295 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.100317 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260928 18:00:52.100442 27965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:40983: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
I20260928 18:00:52.131633 27965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } attempt: 1
I20260928 18:00:52.132220 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.132069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40248)
0928 18:00:52.132147 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.132190 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 18:00:52.132766 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.132683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40264)
0928 18:00:52.132724 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.132750 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 18:00:52.132894 27965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:40983: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
W20260928 18:00:52.142200 28047 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 18:00:52.142493 28047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 18:00:52.142534 28047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 18:00:52.142568 28047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 18:00:52.144030 28047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 18:00:52.144100 28047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 18:00:52.144124 28047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 18:00:52.144155 28047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 18:00:52.144182 28047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 18:00:52.145800 28047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.46.84:45555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.46.126:46663,127.5.46.125:41279,127.5.46.124:40983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/krb5kdc/kudu_127.5.46.124.keytab
--principal=kudu/127.5.46.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.46.124:40983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.46.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 16:40:54 UTC on ccycloud.quasar-hsvjfd.root.comops.site
build id 4677566
I20260928 18:00:52.146116 28047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 18:00:52.146368 28047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 18:00:52.151541 28055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:52.151541 28052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 18:00:52.151547 28053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 18:00:52.151932 28047 server_base.cc:1034] running on GCE node
I20260928 18:00:52.152169 28047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 18:00:52.152437 28047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 18:00:52.153591 28047 hybrid_clock.cc:648] HybridClock initialized: now 1790618452153565 us; error 41 us; skew 500 ppm
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618452, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 18:00:52.156379 28047 init.cc:377] Logged in from keytab as kudu/127.5.46.124@KRBTEST.COM (short username kudu)
I20260928 18:00:52.156904 28047 webserver.cc:460] Webserver started at http://127.5.46.124:40565/ using document root <none> and password file <none>
I20260928 18:00:52.157115 28047 fs_manager.cc:359] Metadata directory not provided
I20260928 18:00:52.157156 28047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 18:00:52.157250 28047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 18:00:52.158305 28047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data/instance:
uuid: "485db5aff4d04a74ba5073f810c0553f"
format_stamp: "Formatted at 2026-09-28 18:00:52 on dist-test-slave-z2t3"
server_key: "8e942549267eb492361ddb9059cd1c22"
server_key_iv: "5aa17f58d19fba046945ef3f5522bf3b"
server_key_version: "encryptionkey@0"
I20260928 18:00:52.158622 28047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal/instance:
uuid: "485db5aff4d04a74ba5073f810c0553f"
format_stamp: "Formatted at 2026-09-28 18:00:52 on dist-test-slave-z2t3"
server_key: "8e942549267eb492361ddb9059cd1c22"
server_key_iv: "5aa17f58d19fba046945ef3f5522bf3b"
server_key_version: "encryptionkey@0"
I20260928 18:00:52.160025 28047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 18:00:52.160735 28062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 18:00:52.160924 28047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 18:00:52.160873 28040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } attempt: 2
I20260928 18:00:52.161031 28047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data,/tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal
uuid: "485db5aff4d04a74ba5073f810c0553f"
format_stamp: "Formatted at 2026-09-28 18:00:52 on dist-test-slave-z2t3"
server_key: "8e942549267eb492361ddb9059cd1c22"
server_key_iv: "5aa17f58d19fba046945ef3f5522bf3b"
server_key_version: "encryptionkey@0"
I20260928 18:00:52.161124 28047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 18:00:52.161320 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.161161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40266)
0928 18:00:52.161237 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.161279 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260928 18:00:52.161648 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.161563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40268)
0928 18:00:52.161602 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.161627 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 18:00:52.161763 28040 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.46.124:40983: Network error: Client connection negotiation failed: client connection to 127.5.46.124:40983: connect: Connection refused (error 111)
I20260928 18:00:52.178107 28047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 18:00:52.180230 28047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 18:00:52.180351 28047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 18:00:52.184522 28047 rpc_server.cc:307] RPC server started. Bound to: 127.5.46.124:40983
I20260928 18:00:52.184600 28114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.46.124:40983 every 8 connection(s)
I20260928 18:00:52.184945 28047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/data/info.pb
I20260928 18:00:52.186000 28115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 18:00:52.187641 28115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:52.191354 5305 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskO6aDtj/build/release/bin/kudu as pid 28047
I20260928 18:00:52.191452 5305 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskO6aDtj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790618177091767-5305-0/minicluster-data/master-2/wal/instance
I20260928 18:00:52.191682 5305 external_mini_cluster.cc:1428] Setting key a4be0f630c549eb81c37f1ba73e73608
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618452, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:52.197976 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.188420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.5.46.126:46663)
0928 18:00:52.188496 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.188498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.188745 (+ 247us) server_negotiation.cc:407] Connection header received
0928 18:00:52.189261 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.189262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.189274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.189335 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.189990 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.190494 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.191710 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.191986 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.192080 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.194668 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.194669 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.194670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.194678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.196308 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.196907 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.196908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.196908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.196963 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.197244 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.197245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.197246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.197380 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.197458 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.197796 (+ 338us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.197881 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 18:00:52.198180 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.188404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.126:46663 (local address 127.0.0.1:48638)
0928 18:00:52.188616 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.188654 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.189130 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.189413 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.189416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.189669 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.189886 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.189890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.190601 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.190602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.191591 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.191594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.192124 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.192126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.192298 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.192998 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.193007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.194574 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.196400 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.196407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.196410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.196818 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.197041 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.197042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.197043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.197165 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.197450 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.197457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.197638 (+ 181us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.197763 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.197932 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":129,"threads_started":1}
I20260928 18:00:52.199115 28115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } has no permanent_uuid. Determining permanent_uuid...
I20260928 18:00:52.201391 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.192165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.5.46.124:40983)
0928 18:00:52.192389 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.192394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.192414 (+ 20us) server_negotiation.cc:407] Connection header received
0928 18:00:52.192442 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.192445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.192474 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.192559 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.193306 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.194183 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.195133 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.195432 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.195520 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.197969 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.197977 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.197977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.197984 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.200181 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.200543 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.200546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.200546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.200610 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.200820 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.200821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.200821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.200964 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.201147 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.201190 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.201279 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":143,"threads_started":1}
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618452, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:52.206565 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.199495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57134 (local address 127.5.46.125:41279)
0928 18:00:52.199561 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.199563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.199599 (+ 36us) server_negotiation.cc:407] Connection header received
0928 18:00:52.199673 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.199674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.199683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.199726 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.200048 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.200498 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.201160 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.201416 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.201477 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.203455 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.203456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.203456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.203463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.204987 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.205372 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.205373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.205373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.205446 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.205659 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.205660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.205660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.205801 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.205888 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.206433 (+ 545us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.206501 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 18:00:52.206641 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.199416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.125:41279 (local address 127.0.0.1:57134)
0928 18:00:52.199549 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.199561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.199614 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.199730 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.199730 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.199931 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.199974 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.199976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.200583 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.200584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.201076 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.201077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.201535 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.201536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.201581 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.201990 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.201996 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.203372 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.205058 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.205059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.205060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.205301 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.205543 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.205543 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.205544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.205599 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.205881 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.205882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.205951 (+ 69us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.206397 (+ 446us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.206500 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113}
I20260928 18:00:52.207104 28115 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.126@KRBTEST.COM
I20260928 18:00:52.208035 27965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } attempt: 2
I20260928 18:00:52.211544 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.202351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48646 (local address 127.5.46.126:46663)
0928 18:00:52.202429 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.202431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.202441 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:52.202496 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.202498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.202509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.202574 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.203732 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.204509 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.205442 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.205734 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.205802 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.208610 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.208611 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.208612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.208619 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.210271 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.210679 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.210680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.210681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.210751 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.210996 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.210997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.210998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.211130 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.211326 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.211363 (+ 37us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.211441 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.126@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618452, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.124@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:52.216830 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.207376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40282)
0928 18:00:52.207424 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.207433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.207468 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.207594 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.207596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.207727 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.207769 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.207771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.208849 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.208850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.209444 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.209446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.210582 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.210583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.210632 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.211696 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.211702 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.213635 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.215573 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.215574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.215574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.215917 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.216105 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.216106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.216106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.216160 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.216478 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.216479 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.216530 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.216591 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.216694 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260928 18:00:52.217042 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.208306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40296)
0928 18:00:52.208359 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.208369 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.208405 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.208677 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.208677 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.208912 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.208953 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.208954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.209923 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.209924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.210522 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.210525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.211063 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.211065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.211130 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.211533 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.211540 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.213061 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.215945 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.215947 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.215947 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.216187 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.216425 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.216427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.216427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.216494 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.216717 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.216718 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.216764 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.216840 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.216942 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":2}
I20260928 18:00:52.217362 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.207423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.5.46.124:40983)
0928 18:00:52.207459 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.207461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.207468 (+ 7us) server_negotiation.cc:407] Connection header received
0928 18:00:52.207525 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.207526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.207535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.207576 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.207901 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.208756 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.209535 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.210500 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.210586 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.213726 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.213727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.213727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.213734 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.215486 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.215982 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.215982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.215983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.216032 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.216243 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.216243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.216244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.216416 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.216487 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.216626 (+ 139us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.217276 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":568,"server-negotiator.queue_time_us":18}
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.46.125@KRBTEST.COM
I20260928 18:00:52.217947 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.208357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.5.46.124:40983)
0928 18:00:52.208518 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.208520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.208527 (+ 7us) server_negotiation.cc:407] Connection header received
0928 18:00:52.208544 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.208561 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.208570 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.208666 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.209065 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.209837 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.210622 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.210961 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.211055 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.213153 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.213154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.213154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.213161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.215820 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.216285 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.216286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.216287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.216348 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.216547 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.216548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.216548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.216661 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.216722 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.217726 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.217790 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260928 18:00:52.219149 28115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: Bootstrap starting.
I20260928 18:00:52.219502 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: Bootstrap starting.
I20260928 18:00:52.219931 28115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:52.220252 28115 log.cc:824] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:52.220293 27965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:52.220605 27965 log.cc:824] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:52.221158 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.212279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.5.46.125:41279)
0928 18:00:52.212329 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.212332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.212341 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:52.212365 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.212366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.212375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.212435 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.212997 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.213735 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.214791 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.215110 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.215186 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.217975 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.217977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.217977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.217986 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.219725 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.220216 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.220217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.220217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.220283 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.220619 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.220620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.220621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.220755 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.220931 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.220969 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.221049 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 18:00:52.221324 27965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: No bootstrap required, opened a new log
I20260928 18:00:52.221565 28115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: No bootstrap required, opened a new log
I20260928 18:00:52.222605 5305 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 18:00:52.223410 27939 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "37691a8e77714144887a617d04d3cdc7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48646
I20260928 18:00:52.223505 27965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.223665 27965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37691a8e77714144887a617d04d3cdc7, State: Initialized, Role: FOLLOWER
I20260928 18:00:52.223628 28115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.223762 28115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485db5aff4d04a74ba5073f810c0553f, State: Initialized, Role: FOLLOWER
I20260928 18:00:52.223755 27965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.223836 28115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.224126 28123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.224175 28115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:52.224179 27965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:52.224200 27939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 18:00:52.224258 28123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.224339 27939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:52.224469 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.224591 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.225302 27939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.225581 27939 leader_election.cc:288] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [CANDIDATE]: Term 1 election: Requested vote from peers 8886a4d9a23746baa34581be2f536696 (127.5.46.125:41279), 485db5aff4d04a74ba5073f810c0553f (127.5.46.124:40983)
W20260928 18:00:52.226161 28145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260928 18:00:52.225978 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37691a8e77714144887a617d04d3cdc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "485db5aff4d04a74ba5073f810c0553f"
I20260928 18:00:52.226317 28089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [term 0 FOLLOWER]: Advancing to term 1
W20260928 18:00:52.226541 28145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:52.227825 28089 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37691a8e77714144887a617d04d3cdc7 in term 1.
I20260928 18:00:52.228169 27915 leader_election.cc:302] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37691a8e77714144887a617d04d3cdc7, 485db5aff4d04a74ba5073f810c0553f; no voters:
I20260928 18:00:52.228298 28123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 18:00:52.228434 28123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [term 1 LEADER]: Becoming Leader. State: Replica: 37691a8e77714144887a617d04d3cdc7, State: Running, Role: LEADER
I20260928 18:00:52.228528 28123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
W20260928 18:00:52.228665 28146 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:52.228704 28146 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:52.228966 28123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37691a8e77714144887a617d04d3cdc7. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.229055 28123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:52.229795 27897 master_runner.cc:186] Error getting master registration for 127.5.46.125:41279: OK, instance_id { permanent_uuid: "8886a4d9a23746baa34581be2f536696" instance_seqno: 1790618452016131 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 18:00:52.232402 27897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 18:00:52.235342 28158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 18:00:52.234264 28047 master_runner.cc:186] Error getting master registration for 127.5.46.125:41279: OK, instance_id { permanent_uuid: "8886a4d9a23746baa34581be2f536696" instance_seqno: 1790618452016131 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 18:00:52.236960 28047 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 18:00:52.237414 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.226949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.5.46.126:46663)
0928 18:00:52.227005 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.227007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.227705 (+ 698us) server_negotiation.cc:407] Connection header received
0928 18:00:52.227804 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.227805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.227818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.227889 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.228684 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.229403 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.230441 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.230719 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.230762 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.232487 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.232489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.232489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.232496 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.234491 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.234904 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.234905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.234906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.234964 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.235169 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.235169 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.235170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.235295 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.235380 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.237228 (+ 1848us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.237320 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 18:00:52.238552 28158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 18:00:52.242444 28089 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [term 1 FOLLOWER]: Refusing update from remote peer 37691a8e77714144887a617d04d3cdc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:52.243098 28123 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 18:00:52.244143 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.227453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57154 (local address 127.5.46.125:41279)
0928 18:00:52.227499 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.227501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.233043 (+ 5542us) server_negotiation.cc:407] Connection header received
0928 18:00:52.233159 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.233161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.233174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.233207 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.237235 (+ 4028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.237996 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.239030 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.239329 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.239402 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.240960 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.240961 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.240962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.240970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.242644 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.243188 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.243189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.243190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.243276 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.243501 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.243502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.243503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.243641 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.243723 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.243959 (+ 236us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.244044 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 18:00:52.244722 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37691a8e77714144887a617d04d3cdc7. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.244864 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.246678 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.227052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.5.46.124:40983)
0928 18:00:52.227109 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.227111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.227512 (+ 401us) server_negotiation.cc:407] Connection header received
0928 18:00:52.227765 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.227766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.227795 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.227827 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.230237 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.230958 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.233320 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.234022 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.234076 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.240142 (+ 6066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.240144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.240145 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.240154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.245381 (+ 5227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.245801 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.245803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.245803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.245875 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.246092 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.246093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.246093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.246228 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.246325 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.246494 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.246582 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 18:00:52.252884 28040 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } attempt: 3
I20260928 18:00:52.254026 28124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.254124 28124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.254268 28123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.254364 28123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:52.257463 28159 mvcc.cc:205] Tried to move back new op lower bound from 7334373180380295168 to 7334373180328890368. Current Snapshot: MvccSnapshot[applied={T|T < 7334373180380295168}]
Sep 28 18:00:52 dist-test-slave-z2t3 krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790618451, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.46.125@KRBTEST.COM for kudu/127.5.46.124@KRBTEST.COM
I20260928 18:00:52.261862 28158 catalog_manager.cc:1347] Generated new cluster ID: 3c4d992ec68b4d65b3187e5ecf410b74
I20260928 18:00:52.261924 28158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 18:00:52.262679 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.253433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.5.46.124:40983)
0928 18:00:52.253474 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.253476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.253485 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:52.253508 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.253509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.253520 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.253580 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.254725 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.255503 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.256522 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.256834 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.256897 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.259597 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 18:00:52.259598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 18:00:52.259599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 18:00:52.259606 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 18:00:52.261362 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.261813 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.261814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.261814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.261893 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 18:00:52.262104 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 18:00:52.262105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 18:00:52.262105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 18:00:52.262240 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 18:00:52.262298 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:52.262507 (+ 209us) server_negotiation.cc:299] Negotiation successful
0928 18:00:52.262590 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 18:00:52.262677 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:52.253180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.46.124:40983 (local address 127.0.0.1:40318)
0928 18:00:52.253235 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 18:00:52.253248 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 18:00:52.253295 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 18:00:52.253633 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 18:00:52.253635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 18:00:52.253840 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0928 18:00:52.253898 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.253900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.255604 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.255605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.256426 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 18:00:52.256429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 18:00:52.256954 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 18:00:52.256955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 18:00:52.257012 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:52.257605 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 18:00:52.257611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 18:00:52.259500 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 18:00:52.261467 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.261468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.261469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.261691 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.261955 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 18:00:52.261956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 18:00:52.261956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 18:00:52.262023 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 18:00:52.262315 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 18:00:52.262316 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 18:00:52.262374 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 18:00:52.262464 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 18:00:52.262572 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260928 18:00:52.264283 28040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: Bootstrap starting.
I20260928 18:00:52.264967 28040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: Neither blocks nor log segments found. Creating new log.
I20260928 18:00:52.265244 28040 log.cc:824] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: Log is configured to *not* fsync() on all Append() calls
I20260928 18:00:52.266219 28040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: No bootstrap required, opened a new log
I20260928 18:00:52.268545 28040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.268679 28040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8886a4d9a23746baa34581be2f536696, State: Initialized, Role: FOLLOWER
I20260928 18:00:52.268762 28040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } }
I20260928 18:00:52.269070 28040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 18:00:52.269160 28013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [term 0 FOLLOWER]: Advancing to term 1
I20260928 18:00:52.270079 28013 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [term 1 FOLLOWER]: Refusing update from remote peer 37691a8e77714144887a617d04d3cdc7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 18:00:52.270632 28163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 18:00:52.270869 28177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 18:00:52.270910 28177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 18:00:52.270274 28014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "37691a8e77714144887a617d04d3cdc7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8886a4d9a23746baa34581be2f536696"
I20260928 18:00:52.273286 28014 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37691a8e77714144887a617d04d3cdc7 in term 1.
I20260928 18:00:52.275723 28166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.275918 28166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.276014 28166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37691a8e77714144887a617d04d3cdc7. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.276118 28166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.276185 28166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.276307 28166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 18:00:52.278070 28180 mvcc.cc:205] Tried to move back new op lower bound from 7334373180380295168 to 7334373180328890368. Current Snapshot: MvccSnapshot[applied={T|T < 7334373180380295168}]
I20260928 18:00:52.280440 28163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "37691a8e77714144887a617d04d3cdc7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37691a8e77714144887a617d04d3cdc7" member_type: VOTER last_known_addr { host: "127.5.46.126" port: 46663 } } peers { permanent_uuid: "8886a4d9a23746baa34581be2f536696" member_type: VOTER last_known_addr { host: "127.5.46.125" port: 41279 } } peers { permanent_uuid: "485db5aff4d04a74ba5073f810c0553f" member_type: VOTER last_known_addr { host: "127.5.46.124" port: 40983 } } }
I20260928 18:00:52.280551 28163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7 [sys.catalog]: This master's current role is: LEADER
I20260928 18:00:52.282334 28158 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 18:00:52.283118 28158 catalog_manager.cc:1504] Loading token signing keys...
I20260928 18:00:52.295754 28158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37691a8e77714144887a617d04d3cdc7: Generated new TSK 0
I20260928 18:00:52.296010 28158 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 18:00:52.325764 27968 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.324521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.5.46.126:46663)
0928 18:00:52.324565 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.324567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.324985 (+ 418us) server_negotiation.cc:407] Connection header received
0928 18:00:52.325145 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.325146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.325159 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.325191 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.325674 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48670: BlockingRecv error: recv got EOF from 127.0.0.1:48670 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 18:00:52.326540 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.324751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.5.46.125:41279)
0928 18:00:52.324788 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.324790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.326010 (+ 1220us) server_negotiation.cc:407] Connection header received
0928 18:00:52.326109 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.326110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.326121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.326151 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.326466 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57162: BlockingRecv error: recv got EOF from 127.0.0.1:57162 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 18:00:52.327289 28120 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.324280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.5.46.124:40983)
0928 18:00:52.324329 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.324331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.326779 (+ 2448us) server_negotiation.cc:407] Connection header received
0928 18:00:52.326879 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.326880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.326892 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.326922 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.327216 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40328: BlockingRecv error: recv got EOF from 127.0.0.1:40328 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 18:00:52.330055 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.329254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.5.46.125:41279)
0928 18:00:52.329290 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.329293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.329512 (+ 219us) server_negotiation.cc:407] Connection header received
0928 18:00:52.329620 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.329621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.329640 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.329669 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.330003 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57172: BlockingRecv error: recv got EOF from 127.0.0.1:57172 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 18:00:52.337169 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.336334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.5.46.125:41279)
0928 18:00:52.336375 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.336377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.336623 (+ 246us) server_negotiation.cc:407] Connection header received
0928 18:00:52.336723 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.336724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.336735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.336779 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.337113 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57178: BlockingRecv error: recv got EOF from 127.0.0.1:57178 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 18:00:52.341223 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.340413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.5.46.125:41279)
0928 18:00:52.340452 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.340454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.340690 (+ 236us) server_negotiation.cc:407] Connection header received
0928 18:00:52.340805 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.340806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.340816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.340846 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.341170 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57190: BlockingRecv error: recv got EOF from 127.0.0.1:57190 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 18:00:52.347178 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.346337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.5.46.125:41279)
0928 18:00:52.346375 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.346377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.346622 (+ 245us) server_negotiation.cc:407] Connection header received
0928 18:00:52.346726 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.346727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.346751 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.346781 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.347120 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57200: BlockingRecv error: recv got EOF from 127.0.0.1:57200 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 18:00:52.357517 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.356528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57212 (local address 127.5.46.125:41279)
0928 18:00:52.356575 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.356577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.356894 (+ 317us) server_negotiation.cc:407] Connection header received
0928 18:00:52.357008 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.357009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.357021 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.357053 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.357453 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57212: BlockingRecv error: recv got EOF from 127.0.0.1:57212 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 18:00:52.376122 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.374954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.5.46.125:41279)
0928 18:00:52.375005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.375008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.375360 (+ 352us) server_negotiation.cc:407] Connection header received
0928 18:00:52.375506 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.375507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.375521 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.375556 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.376053 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57220: BlockingRecv error: recv got EOF from 127.0.0.1:57220 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260928 18:00:52.410517 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.409521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.5.46.125:41279)
0928 18:00:52.409567 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.409569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.409876 (+ 307us) server_negotiation.cc:407] Connection header received
0928 18:00:52.409981 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.409983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.409996 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.410027 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.410448 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57224: BlockingRecv error: recv got EOF from 127.0.0.1:57224 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 18:00:52.476805 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.476063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57228 (local address 127.5.46.125:41279)
0928 18:00:52.476142 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.476144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.476183 (+ 39us) server_negotiation.cc:407] Connection header received
0928 18:00:52.476259 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.476260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.476270 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.476332 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.476738 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57228: BlockingRecv error: recv got EOF from 127.0.0.1:57228 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260928 18:00:52.606930 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.606098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.5.46.125:41279)
0928 18:00:52.606194 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.606197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.606315 (+ 118us) server_negotiation.cc:407] Connection header received
0928 18:00:52.606418 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.606419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.606431 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.606508 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.606878 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57230: BlockingRecv error: recv got EOF from 127.0.0.1:57230 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260928 18:00:52.865056 28044 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 18:00:52.864303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57232 (local address 127.5.46.125:41279)
0928 18:00:52.864373 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:52.864375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:52.864416 (+ 41us) server_negotiation.cc:407] Connection header received
0928 18:00:52.864509 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:52.864510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:52.864525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:52.864591 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 18:00:52.864987 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57232: BlockingRecv error: recv got EOF from 127.0.0.1:57232 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 18:00:53.229147 28146 catalog_manager.cc:1259] Loaded cluster ID: 3c4d992ec68b4d65b3187e5ecf410b74
I20260928 18:00:53.229233 28146 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: loading cluster ID for follower catalog manager: success
I20260928 18:00:53.230100 28146 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: acquiring CA information for follower catalog manager: success
I20260928 18:00:53.230444 28146 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 485db5aff4d04a74ba5073f810c0553f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 18:00:53.271332 28177 catalog_manager.cc:1259] Loaded cluster ID: 3c4d992ec68b4d65b3187e5ecf410b74
I20260928 18:00:53.271414 28177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: loading cluster ID for follower catalog manager: success
I20260928 18:00:53.272310 28177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: acquiring CA information for follower catalog manager: success
I20260928 18:00:53.272650 28177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8886a4d9a23746baa34581be2f536696: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 18:00:53.381604 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:53.378544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.5.46.125:41279)
0928 18:00:53.378686 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:53.378687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:53.378697 (+ 10us) server_negotiation.cc:407] Connection header received
0928 18:00:53.378729 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:53.378730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:53.378786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:53.378877 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:53.379261 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.379814 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.380681 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.380901 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.380962 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:53.381202 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:53.381320 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:53.381387 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:53.381497 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 18:00:53.381523 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260928 18:00:53.386363 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:53.383680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.5.46.124:40983)
0928 18:00:53.383798 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:53.383800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:53.383825 (+ 25us) server_negotiation.cc:407] Connection header received
0928 18:00:53.383887 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:53.383888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:53.383923 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:53.384026 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:53.384224 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.384711 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.385484 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.385685 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.385800 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:53.386024 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:53.386138 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:53.386211 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:53.386270 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 18:00:53.386291 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260928 18:00:53.391263 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:53.388388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57256 (local address 127.5.46.125:41279)
0928 18:00:53.388440 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:53.388442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:53.388607 (+ 165us) server_negotiation.cc:407] Connection header received
0928 18:00:53.388644 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:53.388645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:53.388657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:53.388712 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:53.388878 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.389617 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390456 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.390759 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390825 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:53.390968 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:53.391007 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:53.391056 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:53.391145 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 18:00:53.391168 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 18:00:53.391332 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:53.388262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.5.46.124:40983)
0928 18:00:53.388361 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:53.388363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:53.388747 (+ 384us) server_negotiation.cc:407] Connection header received
0928 18:00:53.388773 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:53.388774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:53.388785 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:53.388837 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:53.388933 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.389403 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390503 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.390797 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390853 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:53.391051 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:53.391085 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:53.391133 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:53.391227 (+ 94us) server_negotiation.cc:299] Negotiation successful
0928 18:00:53.391250 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260928 18:00:53.392022 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 18:00:53.388243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48678 (local address 127.5.46.126:46663)
0928 18:00:53.388415 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 18:00:53.388417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 18:00:53.388426 (+ 9us) server_negotiation.cc:407] Connection header received
0928 18:00:53.388472 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 18:00:53.388473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 18:00:53.388533 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 18:00:53.388631 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 18:00:53.388826 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.389616 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390638 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 18:00:53.390924 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 18:00:53.390984 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 18:00:53.391603 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 18:00:53.391724 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 18:00:53.391783 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 18:00:53.391927 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 18:00:53.391953 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":69,"threads_started":1}
I20260928 18:00:53.408327 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27897
I20260928 18:00:53.413843 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 27972
I20260928 18:00:53.419118 5305 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskO6aDtj/build/release/bin/kudu with pid 28047
2026-09-28T18:00:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2846 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5697 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276335 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST